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We consider a linear-quadratic deterministic optimal control problem where 
the control takes values in a two-dimensional simplex. The phase portrait of the 
optimal synthesis contains second-order singular extremals and exhibits modes 
of infinite accumulations of switchings in finite time, so-called chattering. We 
prove the presence of an entirely new phenomenon, namely the chaotic behaviour 
of bounded pieces of optimal trajectories. We find the hyperbolic domains 
in the neighbourhood of a homoclinic point and estimate the corresponding 
contraction-extension coefficients. This gives us the possibility to calculate the 
entropy and the Hausdorff dimension of the non-wandering set which appears 
to have a Cantor-like structure as in Smale’s Horseshoe. The dynamics of the 
system is described by a topological Markov chain. In the second part it is 
shown that this behaviour is generic for piece-wise smooth Hamiltonian systems 
in the vicinity of a junction of three discontinuity hyper-surface strata. 


Contents 

1 Introduction 

2 Simplest properties of the model control problem 

2.1 Formulation of the model problem. 

2.2 Symmetries of the problem . 

2.3 A theorem on the passage to the adjoint variables . 

2.4 The quotient spaces M/g and M/g . 

x The work was supported by the RFBR (grant N 14-01-00784) 


a 

5 

5 

6 
7 

M 


i 











2.5 One-dimensional Fuller problems in the interior. 

2.6 Barycentric coordinates for the case of an equilateral triangle . 

2.7 Most important examples of periodic trajectories on M/g . . . 

2.8 Behaviour of optimal trajectories in the neighbourhood of peri¬ 
odic trajectories. 


17 

18 

20 


24 


3 


First theorem on the chaotic behaviour of trajectories in inte¬ 
gral vortices 

3.1 Statement of the first theorem on the chaotic behaviour of tra¬ 
jectories in the model problem . 

3.2 Blowup of the singularity at the vertex of an integral vortex . . . 

3.3 Reparametrization of time. 

3.4 Robustness of the self-similar trajectories . 

3.5 Homoclinic orbit on the zero section of the cylinder C . 

3.6 Conclusion of the proof of the first theorem on the chaotic behaviour 


26 


26 

27 

29 

32 

35 

37 


4 Topological properties of the Poincare return map 

4.1 Topological structure of the switching surface. 

4.2 Factorization with respect to the action of the group S 3 

4.3 Known pieces of the synthesis. 

4.4 Global structure of the Poincare map. 

4.5 Transitions of rotational type and transpositional type 

4.6 An attractor in the backward time direction. 

4.7 Intermediate regions. 

4.8 Resolution of the dynamics on region V. 

4.9 Resolution of the dynamics of transitions of type B . . 

4.10 Dynamics of the transitions of types A and C . 


So 

40 

42 

43 

45 

46 

47 

48 
52 
55 
57 


5 Chaotic dynamics of the Poincare map 

5.1 Bilipsclritz property of the Poincare return map. 

5.2 Conditions of Lipschitz hyperbolicity. 

5.3 Conjugation with the topological Markov chain. 

5.4 Bounds on the dimensions. 

5.5 Unilateral Markov chain. 

5.6 The fractal nature of the Poincare map . 

5.7 Main theorem on the chaotic nature of the optimal synthesis in 

the model problem. 


|59 

59 

61 

66 

66 

68 

69 


72 


6 


Chaos on finite time intervals in Hamiltonian systems with dis¬ 
continuous right-hand side 

6.1 Hamiltonian systems with discontinuous right-hand side. 

6.2 Formulation of theorems on chaos in Hamiltonian systems with 

discontinuous right-hand side. 

6.3 Descending system of Poisson brackets. 

6.4 Blow-up of the singularity at the strange point. 

6.5 Model optimal control problem on the zero section Cq . 


|T6 

[76 


77 

81 

85 

88 


2 











































6.6 

6.7 

6.8 


Nilpotentization in the neighbourhood of the strange point . . 

Poincare map in the Hamiltonian system. 

finalization of the proof of the chaotic behaviour of the trajecto¬ 
ries in a generic Hamiltonian system. 


97 


1 Introduction 

The main tool for solving deterministic optimal control problems is the Pon- 
tryagin Maximum Principle (PMP) [15] , It allows one to reduce the control 
problem to a two-point boundary value problem of Hamiltonian dynamics. 
Suppose the control variable u takes values in some set Q. Then the Hamil¬ 
tonian function H defining the dynamics is given as the maximum H(t,x,p) = 
maxugn T~L(t, x,p, u) over the control u of the Pontryagin function TL , and the 
optimal control u(t,x,p), if it exists, is found among the maximizers. 

In general, the maximizer is unique in an open dense subset of the space 
of variables t,x,p and depends smoothly on these variables. On this set, the 
Hamiltonian H will be smooth, whereas on its boundary the derivatives of H 
will experience discontinuities. Usually the Hamiltonian system as a whole will 
be piece-wise smooth on the extended phase space of the variables t,x,p. The 
extended phase space will be divided into disjoint domains Ai ,..., Ak on which 
the Hamiltonian is given by smooth functions Hi,, respectively. The 
dynamics is described by an ordinary differential equation with discontinuous 
right-hand side. We consider situations when the set Q is a convex polyhedron, 
and the domains Ai are those regions where the optimal control resides in a 
particular vertex v l of the polyhedron. The set of points where the derivatives 
of H are discontinuous is a stratified manifold, and on each stratum the optimal 
control is confined to a particular face of the polyhedron ft. 

A trajectory of the Hamiltonian system evolving inside a smoothness do¬ 
main is called regular. If a trajectory passes from one smoothness domain Ai 
into another one Aj , then the corresponding optimal control will experience a 
jump from the vertex Vi of the polyhedron 11 to the vertex Vj. This process is 
called switching , and the discontinuity hyper-surface is called switching surface. 
Typically optimal trajectories intersect the switching surface transversally, in 
which case they are called bang-bang trajectories. It may happen, however, that 
a trajectory moves along the switching surface, in which case one speaks of a 
singular trajectory. Typically uniqueness of the solution does not hold in the 
vicinity of a singular trajectory, and many regular trajectories can join or leave 
from the same singular point. This is possible because the right-hand side of 
the underlying ordinary differential equation experiences a discontinuity. 

For a singular trajectory lying on a switching hyper-surfac£0 one can define 
an order , in dependence on up to which order the Poisson brackets of the ad¬ 
joining smooth pieces Hi of the Hamiltonian vanish. The order may be local or 
intrinsic , in dependence on whether the brackets vanish only on the trajectory 

2 Equivalently one can consider a problem with a one-dimensional control variable in the 
neighbourhood of the singular trajectory [5]. 
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itself or in a neighbourhood of it [7. A good generalization of these notions 
(the so-called natural order) is given in [11] . If the natural order of the singular 
trajectory is even, then a regular trajectory cannot join it in a piece-wise smooth 
manner. In this case regular trajectories spiral around the singular trajectory 
and intersect the switching surface in an infinite number of points in finite time, 
such that the joining point is an accumulation point of switchings. This phe¬ 
nomenon is called chattering , and is well-studied for the situation where exactly 
two smoothness domains meet at the singular trajectory in question [ 13 - 

In this contribution we consider the situation where three smoothness do¬ 
mains A \, A 2 , A 3 meet at a manifold £123 of codimension 2. This situation is 
equivalent to an optimal control problem with 2-dimensional control. The gen¬ 
eral case of singular extremals for n-dimensional control was explored in [15]. 
The role of the order is played by a flag of orders. In the present contribution we 
consider a singular trajectory of intrinsic second order. This work can be seen 
as a continuation of the paper mi, where this case was first considered and 
the presence of the chattering phenomenon was proven. Here we observe an 
additional phenomenon appearing in this concrete problem, namely, the chaotic 
behaviour of bounded parts of optimal trajectories. This phenomenon was not 
yet seen in optimal control problems and is hence entirely new. The key to the 
proof is a new mathematical object: the system of ordinary differential equa¬ 
tions for all Poisson brackets up to the fourth order between the restrictions Hi 
of the Hamiltonian to the domains of continuity Ai neighbouring the singular 
trajectory. We call this system the descending system of Poisson brackets. 

Our findings are not limited to optimal control problems, but rather hold for 
a whole class of piece-wise smooth Hamiltonian systems with three smoothness 
domains joining at surface £123 containing singular trajectories of second order. 
We will consider a particular model linear-quadratic optimal control problem in 
detail whose solution is prototypical for the general case. We call this prob¬ 
lem the model problem for general piece-wise continuous Hamiltonian systems 
because the principal part of the descending system of ODEs for general Hamil¬ 
tonian systems coincides with the descending system of ODEs for the model 
problem. 

Finally, we state some conjectures and possible directions for further work. 

The work consists of two parts. The first part is devoted to the complete 
design of the optimal synthesis for the model problem. The model problem is 
affine in the planar control, the latter takes values in an equilateral triangle 
centered at the origin. It is proved that the optimal synthesis contains an 
entirely new singularity. All optimal trajectories end at the fixed singular point 
in finite time, but nevertheless the control on some extremals has a chaotic 
nature: there exists a topological Markov chain Ep such that the sequence 
of control switchings on an optimal trajectory corresponds one to one to a 
trajectory through a point of Ep under the right Bernoulli shift. The set of non¬ 
wandering points in the model problem has a fractal structure (as in Smale’s 
horseshoe) and non-integer Hausdorff and box dimensions. 

In the second part a generalization of this phenomenon to Hamiltonian sys¬ 
tems with discontinuous right-hand side is proved. The main role in demonstrat- 
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ing the generalization is played by a resolution of singularities for the Poincare 
mapping at the singular point, the so-called strange point. For this purpose we 
introduce an auxiliary descending system of ordinary differential equations on 
Poisson brackets of restrictions of the Hamiltonian to the domains of smooth¬ 
ness neighbouring the strange point. It turns out that the principal part of the 
descending system coincides with the equations of the model problem that was 
investigated in the first part. 

The discovered phenomenon appears in the neighbourhood of a generic sin¬ 
gularity. Namely, in the second part we prove a theorem on the structural 
stability of the synthesis and determine the codimension of the manifold of 
singularities in question. 


2 Simplest properties of the model control prob¬ 
lem 

In this section the model optimal control problem will be formulated and its 
basic properties described. The behaviour of the optimal trajectories of the 
problem is of chaotic nature. It is worth to note that the model problem pos¬ 
sesses a rich group of symmetries, and hence its optimal synthesis can be studied 
to a sufficiently full extent. But at the same time it models the optimal synthesis 
for a broad class of Hamiltonian systems with discontinuous right-hand side. In 
the subsequent sections we show that the behaviour of optimal trajectories for 
systems in this class is similar to the optimal synthesis of the model problem. 

2.1 Formulation of the model problem 

Consider the optimal control problem 

+ OO 

J(x) = \ f {x(t), x(t)) dt — y inf , . 

o V 1 ! 

x = u; 

with initial data 

z(0) = x 0 , ±(0) = y 0 . (2) 

Here x and u belong to two-dimensional Euclidean space U ~ R 2 equipped with 
the scalar product (•, •), and S2 is a closed triangle with 0 € Int H (such triangles 
will be called admissible). 

Remark 1. The deepest investigation of problem © will be conducted for the 
case when Q is an equilateral triangle centered at the origin. Some results 
related to the chaotic behaviour of the optimal trajectories will be obtained also 
for the more general case when Q is not necessarily equilateral. 

Denote y = x. Let <f >, ip be the adjoint variables to x and y as prescribed by 
the PMP. For brevity we shall sometimes write q = ( x,y) and p = (<p,ip). We 
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employ the notation M = U ®U = {(x,y)} for phase space, and the notation 
M =T*M = {(p,q)} for extended phase space. The Pontryagin function has 
the form 


n = -y(x,x) + + (ip,u), 

i.e., 4> = AoX and ip = —<t>. If Ao = 0, then the function ip is linear, and we 
immediately obtain a contradiction with the finiteness of integral ©• Setting 
Ao = 1 we obtain the following Hamiltonian system with discontinuous right- 
hand side: 


J ^ <P = x; x = y, y = u. 

I (ip, u) — > max 1 ' 

{ N ' u£f! 

In problem ([3]) there exists a unique trajectory which is singular with respect 
to H, namely x = y = (p = ip=u = 0. This trajectory is of second order ( jT51b 
In the second part of the paper we show that the behaviour of the optimal 
trajectories of problem © in the vicinity of this singular trajectory (i.e., of 
the origin) is typical for Hamiltonian systems with discontinuous right-hand 
side. This motivates us to investigate the optimal synthesis of problem © to a 
maximal extent. 

2.2 Symmetries of the problem 

The Hamiltonian system © has two important groups of symmetries. The 
first one is the scale group K \ 0. Being one-dimensional, it allows to reduce 
the dimension of the phase space by one. The second group is the discrete 
permutation group S3. It appears only in the case when H is an equilateral 
triangle and allows to prove the chaotic behaviour of the optimal trajectories. 

1. Let A € R\ 0. Consider the following transformation g( A) € GL(8 , M) of 
the extended phase space M\ 

x 1 ^ A 2 x, y i->- A y, (p i-» A 3 </>, ip i->- \ A ip. 

The transformation g( A) changes the Hamiltonian H by a factor of A 4 and 
the symplectic form oj = dp A dq = d(p A dx + dip A dy by a factor of A 5 . 
Hence g( A) takes the set of trajectories of system © onto itself, but with 
a change in velocities by a factor of A. If A > 0, then optimal trajectories 
are mapped to optimal ones, because the functional J changes by a factor 
of A 5 . 

2. If the triangle H is equilateral, then there exists a discrete subgroup 
S3 C 0(2, M) of the orthogonal group of transformations of the plane 
U which preserves the triangle il. The simultaneous action of S3 on the 
vectors x, y, <p and ip defines a transformation of the extended phase space 
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that preserves the set of optimal trajectories, the Hamiltonian H and the 
symplectic structure ui. 

2.3 A theorem on the passage to the adjoint variables 

One of the main tools for designing the optimal synthesis of problem CD is the 
following theoremQjon the passage to the adjoint variables. This theorem allows 
to describe the structure of the manifold M + C M. = T*M comprising those 
trajectories of system Q which are optimal for problem ([]]). 

Denote q = (x, y) £ M = R 4 = U ® U and p = (</>, if) £ TqM = R 4 . 

Theorem 1. Let Q be a convex compact set (not necessarily a triangle), and 
suppose 0 £ Intfh Then for any initial point qo = ( Xo,yo ) the following holds: 

1. There exists a unique optimal trajectory q(t,qo) of problem (OP with initial 
point qo. The adjoint function p(t, qo) is also unique. 

2. There exists a time instant T(qo) > 0 such that q(t, qo) = 0 and p(t , qo) = 0 
for t > T(qo), and q(t,qo) ^ 0, p(t,qo) 0 for t < T(qo). The function 
T(-) is continuous and 

Ci max{\Zfxo[, |y 0 |} < T(q 0 ) < C 2 max{-v/i®oi, |j/o|} 
for some constants C 1 > 0 and C 2 > 0 . 

3. The mapping E : qo H► p(0,qo) = (4>o,il>o) locally Lipschitz and bijective. 

Moreover, 

C3 max{-\/[iroj, |y 0 |} < max{-^o|, < C 4 max{ I 2 / 0 I} 

for some constants C 3 > 0 and C 4 > 0. 

The proof of the theorem consists of several lemmas. 

Lemma 1. For any qo there exists a unique optimal solution q(t,qo) of problem 
m with initial condition qo- 

Proof. Since for any measurable control u(t) 6 D there exists a unique trajec¬ 
tory q(t) with initial condition g( 0 ) = qo, we can consider J as a functional 
on the space of controls Loo(0, + 00 ). The set of all admissible controls U is 
bounded, closed, and convex in Loo( 0 , + 00 ) due to boundedness, closedness, 
and convexity of SI. Hence, due to the Banach-Alaoglu theorem the set U is 
weakly * precompact (Loo(0,+ 00 ) = L*(0,+ 00 )). The weak * closedness of U 
follows trivially from the convexity and from the compactness of SI. Let us show 
that the functional J is lower semi-continuous relative to the weak * topology. 
Consider the operator I\ : Loo(0, + 00 ) —> AC[ 0, + 00 ) given by 
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x(t) = (Ku) (t) = xo + yot + f (t t)u[t) dr. (4) 

Jo 

The operator K takes the control u(t) to the corresponding solution x(t). 
Hence we have to prove that for any sequence of controls u n u one has 


r+oo p+oo 

liminf / ((Ku n )(t), (Ku n )(t)) dt > / ((Ku)(t), (Ku)(t)) dt. 

ri->+oo J g J g 

It is possible to rewrite & in the form ( Ku)(t ) = x 0 + y 0 t + f^°°(t - 
T)u(r)0(t — t) dr, 

where &(•) is the Heavyside function. Since 8 (t — t) £ ii(0,+oo), we have 
that ( Ku n )(t ) converges point-wise to (Ku)(t). The desired inequality follows 
from the Fatou theorem. The uniqueness of the optimal solution follows imme¬ 
diately from the strict convexity of the functional J. □ 

Let us denote the Bellman function by B(qo), 

r 1 f + °° 

B{qo) = B(xo,yo) = inf | -j (x(t),x(t)) dt, 

where x £ Cl and x(0) = xo, x(0) = 2 /o|- 

Obviously 0 < B(qo) < +c» for any go- 
Lemma 2. The Bellman function B is strictly convex. 

Proof. Let x 1 (t) and x 2 (t) be two admissible trajectories. For any A £ R we 
have after some calculus 


j( Ax 1 + (1 — A)x 2 ) = AJ(x 1 ) + (1 — A) J(x 2 ) — A(1 — A )J(x 2 — x 1 ). 

If x 1 (t) and x 2 (t) are optimal trajectories with initial conditions gg = (xj, y}f) 
and g^ = (xg, j/g) respectively, then in view of the nonnegativity of J we obtain 
for 0 < A < 1 that 


+ (1 - A )B{ql) > J(Ax 1 + (1 - A)x 2 ) > B(Ag^ + (1 - A)gg). 


Equality can be attained only if J(x 2 — x 1 ) = 0, i.e., if x 1 = x 2 . 


□ 


Corollary 1. The Bellman function B is continuous. 
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Corollary 2. The mapping qo H > u(-,qo) £ ioo(0, +oo) taking the point qo t° 
the optimal control w(-,go) on the trajectory through this point is continuous in 
the weak * topology. 

Proof. Consider a sequence qk which converges to the point qo. Denote by 
itfc(-) = u(-, qk) the optimal control on the trajectory through the point qk■ Let 
us show that the sequence Uk(-) converges in the weak * topology to the optimal 
control out of the point go- We may select from the sequence Uk a weakly* 
converging subsequence Uk m , because the set U is weakly * compact. We then 

have Uk m Uo- Let us prove that the trajectory xo(t) = JC(uo(-), qo)(t) 
with control uo(-) is optimal for the initial point qo- Indeed, denote by Xk(t) = 
IC(uk{-),qk)(t) the optimal trajectory for the point qk- Then from the continuity 
of the function B and from the Fatou theorem we obtain 

B{q 0 ) = lim B(q km ) = lim J(x km ) > J(x 0 ). 

m—> oo m—too 

Hence the trajectory xo(-) is optimal. Therefore there exists a unique limit of 
the sequence Uk , namely the optimal control u o for the initial point go, i.e., the 
sequence Uk converges in the weak * topology to Uq. 

□ 

Corollary 3. The mapping qo H > q(-, qo) £ C'[0, r] is continuous for any r > 0. 

Proof. Let us show that the mapping go K y x(-,qo) £ C[0,t] is continuous. 

Consider a converging sequence qk —> qo- Then we have u(-,qk) u(-,qo). 

From formula (JU) it follows immediately that x(t,qk) -+ x(t,q 0 ) for all t and 
the convergence is uniform on t 6 [0, r]. Similarly one obtains the proof of the 
convergence y(-, q k ) ->■ y{-, qo)- □ 

Lemma 3. For any initial data qo there exists a time T(qo) such that q(t , go) = 0 
for all t > T(qo) and T(qo) < C 2 max{-^/|xo|, |S/o |}» where C 2 > 0 is some 
constant. 

Proof. At first we shall prove that the optimal solutions q(t, qo) tend to 0 as 
t —> +00 due to the decrease of the function B. Indeed, suppose the opposite: 
let \q(tk, go)| > £ for some £ > 0 and tk —> 00 as k —> 00 . Since the function 
B is continuous, we have M = min| 9 |_ e S(g) > 0. Therefore B(q) > M for all 
|g| > e due to convexity of the function B (since 0 is its absolute minimum). 
So, B(q(tk,qo)) > M > 0 for all k. This yields the claimed contradiction. 

Now let us prove the assertion of the lemma itself. Denote 

p{x,y) = max{y/|a:| /a, \y\/b}, 

where the constants a > 0 and b > 0 will be chosen later. Let p(xo,yo) = b. 
Since the optimal trajectory tends to the origin, it intersects the boundary of 
the ball p(x,y) = ^8 in finite time. The remaining part of the proof will be 
devoted to the estimation of this time. Namely, our aim is to correlate the 
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time of halving p(x, y) on the optimal trajectory and the value of the Bellman 
function. 

We begin with the estimation from below of the integral of (x(t),x(t)') along 
an optimal trajectory x(t) on an interval [0,5]. We use the formula (0]): if 
|xo| = ad 2 , then we have as t £ [0, 5] 

|x(t)| > max{0, ad 2 — bdt — i d 2 max |u|}, 
and if |z/o | = bd, then 

|x(t)| > max{0, bdt — ad 2 — ^ d 2 max |u|}. 

Let us denote c = imax|u|, and put a = 5c, b = 12 c, and r = t/d £ [0,1]. 
Then one of the two inequalities 


|£(t)| = |x(r5)| > 


cd 2 max{0,4 — 12r}, if |x 0 | = ad 2 , 
cd 2 max{0,12r — 6}, if \yo\ = bd 


is fulfilled. In any case either for r £ [0, |], or for r g [f,l] we obtain a non¬ 
degenerated lower bound on |ai(i)|. Therefore for some constant c\ > 0 we 
have 


f (x(t), x(t))dt = d ( (x(rd), a?(r5))dr > Ci<5 5 . 

Jo Jo 


(5) 


Now let p(qo) = p{xo,Vo) = A. Denote for brevity p(t) = p(x(t),y(t)). 
Let ti(qo) > 0 be the minimal time instant such that p(ti(qo)) = ^A = d. 
Calculate successively 


to — O', ti—to + Ti; t 2 —ti+T 2 ; ...; t n+ i — t n + r n+ i; 

n=p(to); T2=p{t 1 y, T 3 = p(t 2 y, ...; r„+i=p(t n ); 

Let N be a number such that t^—i < ti(qo) and tjv > ti(qo). The number N 
is finite because t n +i — t n > d for t n < fi(go)- Since r n > d for n < N, we 
obtain by virtue of inequality ([5]) 


rt N * 

B(q 0 ) > / (x{t,qo),x{t,qo)) dt> = 

Jo k =l 

N N 

= Clf E(|) > ci5 5 ^ y > ci5 4 ti( 9o )- 

fc =1 k =1 

Therefore 


t i(A) < c 2 
2 ' 7 


/3(A) 
A 4 ’ 
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where t i(A) = sup p(go)=A 1i {q 0 ) and /3(A) = max p( , o)=A B{q 0 ). 

Now note that ti(AA) = Afi(A) and /3(AA) = A 5 /3(A) for all A > 0 by 
virtue of the symmetry subgroup g of the group R+. Hence 

T(q 0 ) < (1 + i i + .. ■)ti ( p(q 0 )) = c 3 p{q 0 ), 
which completes the proof. 

□ 

In what follows let us denote by T(qo) the minimal time instant such that 
q{t,qo) = 0 for t > T(q 0 ). 

Remark 2. A lower bound on the time of hitting the origin of the form T(qo) > 
Ci max{y'pEo|, |j/o|} follows immediately from the consideration of the time op¬ 
timal problem 


T —*■ inf as x = y, y = u, u £ x(0) = x 0 , y(0) = y 0 , x{T) = y(T) = 0, 

where Q C and is a square centered at the origin and with side length 
2diamfi. 

Corollary 4. The function T(qo ) is continuous. 

Proof. The continuity of the function T(qo) at qo = 0 follows from the Squeeze 
theorem. Now consider the case qo 7^ 0- Consider a converging sequence qk 
qo- Denote 


T = liminf T(qfc), T + = limsupT(qfc). 

k—loo k—loo 

Note that T(q 0 ) < T~. Indeed, if t > T~, then due to Corollary [3] we have 

q(t,qo) = 0. 

Let us show that T(q 0 ) > T + . According to Corollary [3] we have 
q( T {qo),qk) -> q{T(q 0 ),q 0 ) = o. 

This implies that the optimal trajectories with initial points qk appear at an 
arbitrarily small neighbourhood of the origin after the time T(qo). But at the 
origin the function T(-) is continuous. Hence T(q(T(qo ), qk)) —t 0. It remains 
to note that for any k either T{q k ) < T(q 0 ), or T(q k ) = T(q 0 )+T(q(T(q 0 ),q k )), 
which completes the proof. 

□ 


Corollary 5. For every qo the adjoint function p(t 1 qo) is unique. 

Proof. Since u(t, qo) = 0 for t > T(qo), we have that (if, u) attains its maximum 
at an interior point of H (e.g., 0 £ Int H). Hence if(t, qo) = 0 for t> T(qo), and 
consequently, <f(t,qo) = 0 for t > T(qo). For t < T(qo) the function p(t,qo) is 
uniquely determined by 
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<H.t, 9 o) = - x(t, qo) dr, , g 

q 0 ) = / t Ti ' 9o) 0 ( t , q 0 ) dr. 

□ 


Lemma 4. The Bellman function is of class C 1 and B'(qo) = — p(0,qo). 

Here and in the sequel by B'(qo) £ U* we denote the differential of the 
Bellman function at the point qo- 

Proof. Let us start with the case qo 0. We shall prove that the differential 
of the Bellman function consists of a unique point. This will immediately yield 
B £ C 1 outside of the origin. Consider problem m with the condition q( 0 ) = qo 
replaced by the condition q( 0 ) £ l, where l is any supporting hyperplane to the 
set {q : B(q) < B(qo)} at the point q 0 . By virtue of the strict convexity of the 
function B(q) the trajectory q(t,qo) is also the solution of the new problem for 
any choice of l. In accordance with the Pontryagin Maximum Principle there 
exists an adjoint function p l (t) satisfying equations d3j) and the transversality 
conditions p l ( 0) _L l. By a reasoning similar to the proof of Corollary [5] we 
obtain that p\t) = p(t,qo). Hence the vector p(0,qo) is orthogonal to any 
hyperplane l. If p(0,qo) 7 ^ 0, then the surface level of the function B is smooth 
and the differentiability of the Bellman function B follows from its homogeneity, 
B(g(X)q) = A 5 B(q) for all A > 0. In this case p(0, qo) = X(qo) B'(qo), where \{qo) 
is the coefficient of proportionality. Let us show that p( 0, qo) ^ 0. Assume the 
contrary. Then 0 £ dB(qo), since the sub-differential of a convex function is 
upper semi-continuous. But the Bellman function is strictly convex and has its 
minimum at 0. This leads to a contradiction. 

Now let us show that X(qo) = —1. In accordance with the Bellman equation 

^(xo,a;o) + (B' x {q o ),y(0,q o )) + (B' y (q o ),u{0,q o )) = 0. 

On the other hand, since H = 0, and q(t,qo) and p(t,qo) vanish for t > T(qo), 
we must have TL = 0. It follows that 

~^{x 0 ,x 0 ) + ($(0,q o ),y(0,qo)) + (^( 0 ,go),u( 0 ,g o )) = 0 . 

Hence A(go) = — 1 and B'(qo) = —p(0,qo) for q 0 0 . 

Let us show the differentiability at 0. Let C = vasoi{B{x 1 y)\\x\ 2 + \y \ 4 = 1}. 
Then from B(g(X)q) = A 5 B(q) for all A > 0 we obtain that 0 < B(q) < Cd^l 2 + 
|y| 4 )i. This completes the proof. □ 

Lemma 5. The mapping E : qo 1 —> p(0,qo) = — B 1 (qo) is a homeomorphism 
(i.e., a continuous bijection). 
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Proof. The mapping E is continuous since B £ C 1 . Injectivity follows from 
the strict convexity of the function B. To prove surjectivity we consider the 
strictly convex set B\ = {q : B(q) < 1}, which has a smooth boundary. For any 
direction v € U* there exist exactly two distinct tangent hyperplanes to dB\ 
which are perpendicular to v. Hence the differentials of the function B(q) at the 
contact points q± and (72 are anti-parallel. Therefore in one of these points the 
differential of the function B(q) is anti-parallel to v, say v = —\B'(qf) for some 
A > 0. It follows that v = — B' (g(yf\)q{). 

□ 

Remark 3. By the definition of T(q 3 ) we have q(t, go) 7^ 0 for all t < T(qo). Since 
the mapping E is a homeomorphism and £7(0, 0) = (0,0), we have p(t,q 3 ) ^ 0 
for all t < T(qo). 

Corollary 6. At the initial point the adjoint variables satisfy the bounds 


C 3 max{y/[xoj, | j/o |} < max{ y/\fo\, {/[*/>oT} < C 4 maxl-y/M, I 2 / 0 I} 
for some C 3 > 0 and C 4 > 0. 

Proof. We use the fact that E : q 0 i-» p( 0, go) is continuous. Denote 
C 3 = min maxiyf^i, ffiM}, 

max {•\/l®ol,|j/o |}= 1 

£ 4 = max max{-y/[ 0 of, -y/f^ol}- 

max{- v /|a:o|,|yo|}=l 

Clearly C 4 > C 3 > 0. But £7 “ 1 (0,0) = (0,0) by LemmaU It remains to make 
use of the action of the group g: 

C 3 X < max{< C 4 A 

for max{-^/proj, I 2 / 0 1} = A, which completes the proof. □ 

It remains to prove that the mapping E is Lipschitz. The proof is divided 
in two lemmas. 

Lemma 6. For every point q 3 € U there exist such S > 0 and C > 0 that for 
any Aq = (Ax, Ay), |Ag| < S the inequalities 

0 < B(q 0 + Aq) - B(q 0 ) - B'(q 0 )Aq) < C\Aq \ 2 

hold. Here the constants C and S can be chosen as continuous functions of go. 

Proof. For t £ [0;T(go)], define q(t) = (x(t),y(t)) and p(t) = (</>(t), ip(t)) as the 
solutions of the ODE 


y(t) = u(t, g 0 ), 
x{t) = y(t), 
f)(t) = x(t), 
1 p(t) = -<t>(t), 


2/(0) = 2/0 + Ay, 
ir(0) = xq + Ax, 
<t>{ 0) = ?(0, go), 

^(0) = 2 ?( 0 , g 0 ). 
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For t > T(qo) the trajectory q(t) coincides with the optimal one with initial 
point q(T(q 0 )), i.e., q(t) = q(t - T (q 0 ), q(T (q 0 ))) (the trajectory p(t) is not 
important for t > T(qo)). In this way, the trajectory q(t) begins at qo + Aq and 
moves with the control that is optimal for the initial point qo until this control 
becomes identically zero, then the trajectory switches to the real optimal control. 
Integrating two times by parts, we obtain 

l o) 

B(q 0 + A q) - B(q 0 ) < - / (x(t) - x(t, q 0 ), x(t) + x(t, q 0 )} dt+ 

+B{Ax + T(q 0 )Ay , Ay) 

= ^(<t>(t) + 0(0 do), Ax + tAy)\g iqo) + i(0(i) + 0(0 do), Ay)\^ (qo) + 
+B( Ax + T(q 0 )Ay, Ay) 

= ^(^(T(q 0 )),Ax + T{q 0 )Ay) - (0(0, q 0 ), Ax) + ^(ip(T(q 0 )), Ay) - 

-(•0(0, qo), Ay) + B{Ax + T(q 0 )Ay, Ay) 

= ^||A^|| 2 + ^M(Ay,Ax) + ^MllAyH 2 + (B'(q 0 ),A q ) + 

+B{Ax + T(q 0 )Ay , Ay). 


Here we used that 0(0) = 0(0, qo) = -B' x (q 0 ), <j>(T(q 0 )) = T(q 0 )Ax + 
-Ay , 0(0) = 0(0, g 0 ) = -B' y (q 0 ), 0(T(q o )) = -^-^-Ax - ^-^-Ay, 
p{T(qo),qo) = 0. Introduce the constant C 5 = ma x{B(q) 11|a;|| 2 + ||y || 5 = 1}. 
Then B(Ax + T(q 0 )Ay, Ay) < C^dl Ax + T(q 0 )Ay\\^ + || Ay|| 5 ). We obtain the 
bound 

B( qo + Aq)-B{qo)-(B\qo),A q )<C 6 \\A q \\ 2 + C l 5 (\\Ax\\^ + \\A y \\l + \\A y \\ 5 ), 

where the constants C' 5 ,Ce > 0, generally speaking, depend on T(q 0 ). The 
obtained inequality gives the upper bound claimed in the lemma. The convexity 
of the Bellman function B(q) implies that this expression is nonnegative. □ 

The main difficulty in the remaining part of the proof, i.e., that the function 
B' (q) satisfies a Lipschitz condition, is that this function may not be twice 
differentiable (this is indeed the case if, say, SI is a polyhedron). We shall 
conclude the proof of Theorem [T] with the following lemma. 

Lemma 7. Let g £ C 1 (R n , R) be an arbitrary function. If for some constants 
0 < C' < C and for all < 71,(72 in some open convex domain in R" we have the 
inequality 


C'\Aq 2 \ < g{q 2 ) - g{q 1 ) - (g\ qi ),Aq) < CjAg 2 |, 

where Aq = q 2 — qi, then the mapping g’ : R” —> R"* satisfies a Lipschitz 
condition with Lipschitz constant C. If moreover, C > 0, then the mapping g' 
is bi-Lipschitz on this domain. 
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Proof. Without loss of generality we can assume that the convex open domain 
is 1". We may also assume that a scalar product is defined in R n and hence 
g'(q ) £ R”. Let to(q) > 0 be a convolution kernel, more precisely, a nonnegative 
C°° function with compact support suppw C {|g| < 1} and f u>(q)dq = 1. 
Denote u>k(q) = k n oj(kq) and g k = g * oj k , where * denotes the convolution. 
Then g k is C°° , g' k = g' * oj k , and 


Ante,92) = gfcte) - 9 k(qi ) - site) A <7 = 

= f Rn {g(q2 - y)-g(qi - y) -g'{qi - y)Aq)uj k {y)dy. 


Consequently, 

C"|Ag| 2 < A k (q 1 ,q 2 ) = ^si'te)[ A 9; Ag] + o(|Ag| 2 ) < C|Ag| 2 . 

By substituting AAg for Ag and by letting A —> 0 we obtain that for all k, q , A q 
the inequality 


C'|Ag| 2 <i 5 "(g)[Ag;Ag]<C|Ag| 2 . 
holds. Hence for all k and g we have ||gl , (g)|| < 2 C. It follows that 

Isite) - flfcte)l < sup |5fc(gi + $te ~ <7i))||Ag| < 2C]Ag|. 

6>e[o,i] 

By letting k —> 00 (while g! and g2 remain fixed) we obtain the desired result 

tete) -g'(qi)\ < 2C|Ag|. 

Now let C' > 0. We shall show that the mapping g satisfies a bi-Lipschitz 
condition. We have 


Hence 


site) - site) = / g"(qi + tAq)Aqdt. 


(site) -g'k(qi), Ag) 



flfcte + iAg)[Ag, Ag] (it 


= fo Si'te + ^Ag)[Ag, Ag] dt > 2C"|Ag| 2 . 


So, for any k,q 2 ,qi we get 

j(flite) -site)> A 9 >| > 2C' / |Ag| 2 . 
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By letting k —> oo (while q\ and q -2 remain fixed) we obtain 


|<<? , (<72)-</(<?i),Ag>| >2C'|Ag| 2 . 

Taking into account that | g'(q2) — g'(qi)\ |Ag| > \(g'(q2 ) — g'(qi), Ag)|, we ob¬ 
tain 


I g\qi) -g'{qi)\ > 2C"|Ag|, 

which means that the mapping g is bi-Lipschitz if C’ > 0. 

□ 

This proves Theorem [T] As a first application of Theorem |T] we obtain (here 
and in the sequel M = {(x, y)} = U © U is the phase space) 

Corollary 7. The set of all pairs ( qo , E(qo)) in the extended phase space M. = 
T*M is a Lipschitz manifold M + which projects bijectively onto the plane {p = 
0}. All optimal trajectories belong to M + and reach the origin in finite time. 
On the other hand, M + consists of optimal trajectories only. There do not exist 
other trajectories of the Hamiltonian system m that reach the origin because 
otherwise they would be optimal due to the convexity of the problem. 

Let us remark that it is not difficult to find trajectories of system © that 
come out of the origin. Indeed, the mapping g(— 1) preserves system ©, but 
reverses the direction of movement. Hence the trajectories that come out of the 
origin form a Lipschitz manifold M_, 

M~ = g(—T)[M + \. 

2.4 The quotient spaces M/g and Ai/g 

A paramount role in the investigation of the optimal synthesis (its existence has 
been proved in Theorem [[]) is played by the transition to the quotient space 
relative to the action g of the group R+, and by the transfer of the optimal 
synthesis to this quotient space. 

Remark 4. Since the above-mentioned actions of both groups (for arbitrary 
triangles fi) and S 3 (for equilateral triangles only) take optimal trajectories into 
optimal ones, the mapping E constructed in Theorem |T] commutes with these 
actions. 


E(x,y) = 


E({\ 2 x, Xy)) 
E(ax, ay) 


(A 3 cj), AV) VA > 0 
aip) Va € S 3 


After the factorization of the phase space M = R 4 (without the origin) with 
respect to the action g of the group R+ we obtain a three-dimensional sphere 
S' 3 . We can correctly transfer the optimal trajectories of problem (HI) to S 3 
with preservation of the direction of movement on these trajectories. However, 
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the velocity on the trajectories will be lost. After factorization of the sphere 
{M \ 0}/g(R + ) = S 3 * with respect to the action of the group S 3 one obtains a 
space similar to a lens space0. One can also factorize the extending phase space 
A4 = T*M = R 8 without the origin with respect to the action g of the group 
R+. In this case one obtains the seven-dimensional sphere S 7 = {.Ad\0}/g(R+). 
We can correctly transfer the trajectories of the Hamiltonian system m onto 
the sphere with preservation of the direction of movement. 

Denote by n the canonic projection M \ 0 —> {M \ 0}/g(R+), and by 7 f the 
canonic projection M \ 0 —> {Ai \ 0}/g(R+). When there will be no risk of 
confusion we shall write 7r instead of 7r. By some abuse of notation, we shall 
write M/g instead of {M\0}/g(R + ) and Ai/g instead of {.M\0}/g(R + ), always 
meaning that the origin is excluded before factorization. 

Theorem [T]allows one to transfer the optimal synthesis from the space M = 
{(£,£/)} onto the space N = {(</>, ip)}- Moreover, using the mapping E we may 
identify M , M + , and N (or, after factorization M/g , M + /g , and N/g). This 
turns out to be very useful while studying the switching surface S of the control, 
because the latter is simplest when described in the space of adjoint variables 
N. Since E commutes with g( A) for A > 0, the spaces M/g , M + /g , and N/g 
can also be identified. 

2.5 One-dimensional Fuller problems in the interior 

In this subsection we consider an important special case, namely when the origin 
lies on one of the altitudes of the triangle D. In this case there exists a two- 
dimensional plane in M such that 

(i) the optimal trajectories of system 0 starting in this plane cannot leave 
it, and 

(ii) the behaviour of optimal trajectories on this plane is equivalent to that 
for a one-dimensional Fuller problem with a non-symmetric interval of control. 

Lemma 8. Let the affine line A containing one of the altitudes offl pass through 
the origin. Then if one of the two following conditions is fulfilled: 

(i) the initial point lies in A® A, i.e., Xo,yo £ A, 

fii) the initial values of the adjoint variables lie in A, i.e., dP, 1 b 0 £ A, where 
(d 0 ,-^ 0 ) = E(x 0 ,y 0 ), 

then the values of the variables on the optimal trajectory do not leave A for all 
t > 0: x(t,qo) £ A, y{t,q 0 ) £ A, 4>(t,q 0 ) £ A, ij)(t,qo) £ A and u{t,q 0 ) £ A 
(recall that qg = ( Xo, yo)). 

Proof. Suppose condition (i) holds. Then by Theorem [T] the optimal trajectory 
q(t , go) exists and is unique. Consider the trajectory q(t) that is obtained by 
orthogonal projection of the optimal trajectory on A® A. The trajectory q(t) 

3 More precisely, one obtains the quotient space of the lens space L(3; 1,1) relative to the 

non-free action of Z 2 . 
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is admissible since A contains an altitude of fi. Since the length of a vector is 
greater than that of its orthogonal projection, we have J(q) < J(q). It remains 
to remark that both q(t ) and q(t,qo) start from the same point. Consequently, 
q(t , qo) = q(t) by virtue of the uniqueness of the optimal trajectory (by Theorem 
[l}. Hence q(t, qo) € A © A for all t. Both adjoint variables <p(t , qo) and ip(t, qo) 
lie in A due to (0. 

By restriction to the subspace A® A C M we obtain a one-dimensional Fuller 
problem, its interval of control being the altitude of the triangle Q contained in 
A. Hence E(A © A) = A ® A (see [13] . §3.5). 

Suppose condition (ii) holds. By Theorem Q] one can find the unique ini¬ 
tial values xq and yo such that E(xo,yo) = ((p^iW)- Therefore xq and yo 
must belong to A (since the mapping E is bijective and E(A © A) = A® A). 
Consequently, condition (i) holds and the claim of the lemma follows from the 
considerations above. □ 

Let us remark that under the conditions of Lemma | 8 ] the singular control 
relative to the base ( ij) of the altitude contained in A is active only on optimal 
trajectories that are contained in A. Indeed, if the maximum of (ip, u) is attained 
simultaneously at all points of the edge (ij), then the vector ip is perpendicular 
to the edge (ij). For a singular control it is hence necessary that the vector ip 
is perpendicular to the edge (ij) during some open interval of time t £ (ti,^)- 
By differentiating ip(t) along the trajectories of system 0 one obtains that 
all vectors <p(t), x(t), y(t) and u(t) are perpendicular to the edge (ij) for this 
interval. 

Definition 1 . We shall call an optimal trajectory semi-singular relative to the 
edge (ij) of the triangle H if the trajectory belongs to the line A containing the 
altitude of H which is perpendicular to the edge (ij). 

2.6 Barycentric coordinates for the case of an equilateral 
triangle 

The proof of the theorems on the chaotic nature of the optimal synthesis relies 
on key elements of the optimal synthesis of problem 0 for the case when the 
triangle H is equilateral. Therefore, the main goal of this subsection will be 
the determination of the key element of the optimal synthesis in this important 
special case. The triangle Q will be supposed to be equilateral and centered on 
the origin (in what follows for the sake of brevity we shall call this case ’’the 
case of an equilateral triangle”). 

To describe explicitly some important examples of optimal trajectories we 
need a suitable coordinate system. Since the set Q is an equilateral triangle 
centered on the origin, the most convenient way is to use the barycentric coor¬ 
dinate system. (Generally speaking, the sum of the coordinates must equal 1 in 
barycentric coordinates. Nevertheless, with a little abuse of notation we shall 
use the term ’’barycentric”.) 
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( x = (a; 1 ,a: 2 ,x 3 ), x 1 + x 2 + x 3 = 0; y = (y 1 ,y 2 ,y 3 ), y 1 + y 2 + y 3 = 0; 

< <i>= (<t> 1, 02, 03), 01 + 02 + 03 = 0; 0 = (01,02,03), 01 +02+03 =0; 

[ u = (u 1 ,u 2 ,u 3 ), u 1 + u 2 + u 3 = 0; Wj < 1,i = 1,2,3. 

The vertices of the triangle fl are the points!! (—2,1,1), (1, —2,1), and (1,1, —2). 
We shall call these the first, the second, and the third vertex of the triangle Cl, 
respectively. In terms of barycentric coordinates, the condition of the maximum 
in ([3|) is rewritten in the form 

u 1 = —2,vP = u k = 1, if 0i < min(0j,0fc), where {i,j, k} = {1,2,3}, 

with the exception of the cases when 0j = 0j < 0/-. 

The action of the discrete group S 3 on the barycentric coordinates consists 
in a permutation of the indices. Let us denote by Aj 3 the line in U that is 
symmetric relative to the transposition (ij ) € S3. By Lemma [8] it follows that 
by restriction to each of the three lines A12, A13, and A23 we obtain a one¬ 
dimensional Fuller problem with non-symmetric interval of control, namely 

+ OO 

f x 2 dt —^ min; 

0 

x = y, y = u\ u e [-•v/6, 5V6] • 

Here x, y, and u are one-dimensional. This problem is well-studied, its opti¬ 
mal synthesis has been designed and can be rewritten as an explicit expression 
of the coordinates, its Bellman function and the mapping E have also been 
found explicitly (see El)- The geometrical properties of the optimal synthesis 
are as follows. On the plane R 2 there exists a one-parametric family of tra¬ 
jectories that is taken onto itself by the action g of the scale group R+. Each 
optimal trajectory reaches the origin in finite time. The optimal control has an 
infinite number of switchings from the left end of the interval to the right end 
and vice versa. After factorization of the phase space R 2 (without the origin) 
with respect to the action of the scale group g one gets the circle S 1 , which at 
the same time is a periodic cycle Z. All optimal trajectories are taken to this 
cycle Z ~ S 1 . The cycle Z is divided in two open intervals Z l and Z r by two 
switching points. The control on Z l is at the left end of the interval and on Z r 
on the right end. The countable number of control switchings on the optimal 
trajectories corresponds to a countable number of rotations along the cycle Z . 

Therefore the mapping of the optimal synthesis of problem m onto the 
quotient space M/g ~ S 3 takes all optimal trajectories from A i3 onto the same 
trajectory Zij ~ S 1 . The trajectory is a periodic cycle which is divided by 
two switching points in two smooth intervals Zf- and Zg. On Z™- the control 

4 We consider the case when the distance from the vertices to the origin equals y/6. Any 
case where the triangle has a different size can be reduced to that particular one be an 
evident change of coordinates. 
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in the vertex k of ff is used, i.e., iq = Uj = 1, Uk = —2, k ^ i,j, and on Z?- 
the midpoint of the edge of (ij) is used, iq = Uj = — i, Ufc = 1. We call the 
first control non-singular and the second one serai-smgida?@ with respect to the 
edge (ij) of the triangle U. 

So, on the quotient space M/g three mutually linked cycles Zij are defined. 
It is easy to check that their mutual linking numbers equal 1 HZ]. 

2.7 Most important examples of periodic trajectories on 

M/g 

A very important role in constructing the optimal synthesis of problem m is 
played by the self-similar trajectories. We use this term for trajectories that are 
periodic up to the action g. 

Definition 2. We call a trajectory (x(t),y(t),(p(t),ip(t)) of system J3]) self¬ 
similar if there exist a time instant to > 0 and a number Aq > 0 such that 


g(\ 0 )(x(t),y(t), <f>(t),ip(t)) = (x(t 0 + X 0 t),y(t 0 + A 0 t),</>(t 0 + X 0 t),ip(t 0 + A 0 t)). 

Remark 5. If Ao < 1, then the self-similar trajectory reaches the origin in finite 
time to; hence by Theorem [1] it lies in M + and is optimal. If Ao > 1, then 
the self-similar trajectory lies in M_ and leaves the origin in finite time with 
an infinite number of switchings. If Ao = 1, then the self-similar trajectory is 
periodic and does not tend to the origin neither as t —> +oo, nor as t —> — oo. 

Since the action g respects the optimal synthesis, we have that the map 
g( A) takes any self-similar trajectory with Ao < 1 to a self-similar trajectory 
with the same Ao- All trajectories of the corresponding family are projected 
onto the same periodic trajectory on the quotient space M + /g. As it will be 
shown below, there exists a countable number of periodic trajectories on M+/g , 
and consequently, there exists a countable number of families of self-similar 
trajectories. 

We shall now describe some specific periodic trajectories on M + /g. The 
proof of the chaotic nature of the optimal synthesis on M + /g will be based on 
this description. We begin with exact definitions of the Poincare first return map 
and of the switching surface. Usually by switching surface one understands the 
set of points where the control on the trajectories of the Pontryagin Maximum 
Principle is discontinuous. But due to the existence of trajectories with semi¬ 
singular control at the midpoints of the edges of f l we extend the definition. 

Definition 3. By switching surface S of problem (HD will denote the set of 
points of the extended phase space M. = T* M where the maximum over u of 
the scalar product (ip, u) in (]3]) is attained at more than one point of the triangle 
Q. That means that 

5 The symbols s and n in the indices Z?- and Z T j stand for singular and non-singular 
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S — S 12 U 523 u 5i3 C A4. 


Here S,j C M is the set of points where argmax u6n (^>, zt) contains the edge 
(ij) of the triangle fl (in this case the co-vector if is perpendicular to the edge 
(ij) of the triangle f l). By 1 S 123 we denote the intersection 

5i23 = 5i2 D 1 S 23 n 5ia = {if = 0} C Ai 
If the triangle f 1 is equilateral, then in barycentric coordinates we have 

Sij = {A = ipj < V’fc}) 

The set S \ 1 S 123 is a smooth co-dimension 1 non-compact manifold without 
boundary. At any point S \ 1S123 the velocity of system m suffers a tangent 
jump. More precisely, at any point of S \ S 123 the difference of the limits of the 
velocities of system ([3]) from both sides of S \ £123 is tangent to S \ 5i23- If the 
velocity at some point <S\Si 23 is transversal to <S\5i23 from both sides, then the 
trajectory of system © which begins at that point does not intersect S during 
a finite interval of time. The Poincare first return map $ is defined on a subset 
of the manifold S \ £123 and takes a point q £ S \ £123 to the first intersection 
of the trajectory of the Hamiltonian system ([3]) starting at q with the switching 
surface S. Denote the image of q by $(q) £ S. If the point q lies on an optimal 
trajectory, then its image $>(q) lies on the same optimal trajectory. Taking into 
account that M + n 5 i 23 = { 0 }, we obtain $((S\Si23)HM + ) C ( 5 \Si 23 )nM + . 
Let us remark that the mapping $ is not defined for all points of (5\Si23)nM + . 

Let us investigate the periodic optimal trajectories on M+ /g for the case of 
an equilateral triangle. Clearly the trajectories Zij are periodic on M+/g. The 
remaining optimal trajectories intersect the switching surface S \ 1S123 transver- 
sally. More precisely, let qo be an initial point and let (aq, z/i, ip 1 , ip 1 ) be the 
intersection point of the optimal trajectory through q 0 with the switching sur¬ 
face S^, taking place at some time instant t\ £ ( 0,T(q o )). Suppose that at 
the intersection point not only the condition i/d = z/d, but also the condition 
(j)\ = </>j holds, which is equivalent to the condition that the intersection is not 
transversal. Then by Lemma [8] the trajectory lies on Aij for t > t\. Hence for 
any optimal trajectory only three cases are possible: (i) on the optimal trajec¬ 
tory holds for all t > 0; (ii) the optimal trajectory 

intersects the switching surface transversal!^ for 0 < t < T(<?o), and (hi) the 
optimal trajectory intersects the switching surface transversally until it reaches 
A^ for some t\ £ (0,T(q o )) at some point (aq,z/i ,(j) 1 ,ip 1 ), and after that it 
remains on A,)j for all t>t\. We have proved the following lemma: 

Lemma 9. All periodic optimal trajectories on M+/g of system 0) with the 
exception of the semi-singular ones (see Lemma\^j define periodic orbits of the 

6 Of course the velocity on the trajectories experiences a jump on the switching surface, 
but its limits from both sides of the switching surface are transversal to it. 
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(a) Three-link cycle Z ± . (b) Four-link cycle Q l (c) Six-link cycle R ± 

Figure 1: Schematic depicting of the periodic trajectories Z ± , Q l , and on 
the sphere M + /g. 


Poincare mapping $ of the switching surface to itself. Moreover, any such 
trajectory intersects the switching surface transversally. 

Let us remark that the arc 7 of the trajectory which joins q with $((7) is 
a smooth curve, and the vectors x{t), y(t), 4>(t), and tp(t) on this arc 7 are 
polynomials in t of second, first, third, and fourth degree, respectively. 

Since the action g respects the Hamiltonian system f3j, it commutes with 
the Poincare mapping $, and it induces a mapping of a subset of (S fl M + )/g 
to itself which with some abuse of notation will also be denoted by <f>. 

Consider a periodic trajectory (i.e., a cycle) on M + /g with k intervals of 
smoothness, i.e., which intersects the quotient S/g of the switching surface 
successively at points zq , Z \, ..., z^-i, Zk = zq- One can find such trajectories 
by considering the system of equations 


$(z 0 ) = zi,...,$(zk-i) = zq •<=> $ fc (2 0 ) = -Zo- 


Lemma 10 ( [17]). There exist the following periodic trajectories on M + /g: 

(i) two three-link cycles Z ± . The control on each of these cycles runs cyclically 
through all three vertices of the triangle SI, clockwise 1 — > 2 — > 3 — > 1 and 
counter-clockwise 1 — > 3 —> 2 — > 1, respectively. 

(ii) three four-link cycles Q l , i = 1,2,3. The control on Q l alternates on the 
vertices of the triangle H in the following order: i^j^i^k-^i, 
where j,k 7^ i. 

(Hi) two six-link cycles . The control on R + runs through all three vertices 
of the triangle H in the order 1—>2—>3—>1—>2—>3—>1, and on R~ 
in the reverse order. 

Proof. We shall detail the proof from m for item (i). It is rather difficult to 
directly solve the equation <k 3 (z) = z, z € S/g, because the arising polynomials 
have too large degrees. To simplify the calculation we will use the discrete group 
of symmetries S 3 of the optimal synthesis, which was described in the subsection 
12.21 Let a = (123) £ S 3 . We assume that the sought cycle is invariant under 
the action of a and we shall find the solution of the corresponding simplified 
equation 
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(a o $) (2) = 2. (7) 

The solution of the simplified equation will be evidently a solution of the initial 
equation <J> 3 (z) = z, because a and $ commute. 

Let some pre-image z £ tt~ 1 (z) have the coordinates z = (xo,yo, <j>°, ip°) and 
denote by 7 the trajectory of system @ in M which passes through 2 at t = 0, 
7(0) = z. Let us now describe the switching of the control. Suppose at the 
point z the control switches from the third vertex of the triangle f l to the first 
one. This implies that 


^ 1=^3 <"02 and ( 8 ) 

Since the control uq is constant on the arc from the point z to the point $(2) 
and given by Uq = (uj, Uq, Uq) = (— 2,1,1), we have that 7 is given by 

! y(t) = yo + u 0 t 

x(t) = x 0 + yot + kuot 2 / q x 

(j>(t) = cjp + x 0 t + \y 0 t 2 + | u 0 t 3 ' 

ip{t) = - 0 ° - <p°t - \x 0 t - \y 0 t 2 - ±u 0 t 3 

on this arc. Suppose the trajectory 7 intersects the switching surface again 
at the time instant to > 0, then a(7r(7(to)))= z, because z satisfies equation 
0 . In other words, there exists a number Ao > 0 such that 0(7(to)) = g(Xo)z. 
This last condition can be rewritten by virtue of <0 as a system of algebraic 
equations that are linear in x°, y°, <j>° , and ip°, and polynomial in t 0 and ^o- 
The solution of this system is defined only up to the action g. This follows from 
the fact that the point z £ 7r _1 (2) is defined only up to the action g too. Hence, 
without loss of generality, one can set to = 1. Solving the resulting system with 
respect to the linearly entering variables cc°, y°, <jr , and ijr and substituting the 
solution into the equality ip 3 = ^3 from 0, we obtain a polynomial equation 
on Ao: 


Pz{ Ao) = A® — 4 Aq - 7Ag - A\l + 1 = 0. 

By the Sturm method we obtain that the polynomial Pz{ Ao) has exactly one 
root in the interval (0,1). This root yields a point z satisfying inequalities 0. 
Hence on M + /g there exists a unique periodic trajectory Z + which is invariant 
with respect to the action of a. The other trajectory Z~ £ M + /g is obtained 
from Z + by application of the reflection (12) £ S 3 or by replacement of the 
permutation a = (123) by a = (132). 

Items (ii) and (iii) are proved similarly to item (i). To obtain the four-link 
cycle on M + /g it is necessary to solve the equation (a o $ 2 )(2) = z, where a is 
one of the three transpositions (12), (23) or (13) from S 3 . The method leads to 
the polynomial equation 
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Pq{ A) = 1 + 4A + 60A 2 + 220A 3 - 607A 4 - 5080A 5 - 19 700A 6 - 73 944A 7 - 

-192 258A 8 - 416 272A 9 - 918 956A 10 - 1609 184A 11 - 2 528 300A 12 - 
-4 868 880A 13 - 5 019 696A 14 - 10 839184A 15 - 8 659 545A 16 - 
-18 568 404A 17 - 12 399 696A 18 - 27180 572A 19 - 14 695 579A 20 
-31988 656A 21 - 16 556 344A 22 - 3 1 98 8 6 56A 23 - 14 695 579A 24 - 
-27180 572A 25 - 12 399 696A 26 - 18 568 404A 27 - 8 6 5 9 5 45A 28 - 
-10 839 184A 29 - 5 0 1 9 6 96A 30 - 4 868 880A 31 - 2 528 300A 32 - 
-1609 184A 33 - 918 956A 34 - 416 272A 35 - 192 2 58A 36 - 73 944A 37 - 
-19 700A 38 - 50 80A 39 - 607A 40 + 2 20A 41 + 60A 42 + 4A 43 + A 44 = 0, 

which also has a unique root in the interval (0,1). The coordinates of the points 
z and $(z) satisfy inequalities which are similar to ©, and consequently, on 
M+/g there exist three four-link periodic trajectories which are invariant with 
respect to a permutation of order 2. 

To obtain the six-link cycles on M + /g it is necessary to solve the equation 
(ao$ 2 )(j) = z, where a is one of the cyclic permutations (123), (132) from S 3 . 
As a result one obtains 


P R { A) = A 20 - 12A 19 + 30A 18 + 66A 17 - 117A 16 - 504A 15 - 207A 14 + 

+942A 13 + 1271A 12 - 390A 11 - 1599A 10 - 390A 9 + 1271A 8 + 942A 7 - 
-207A 6 - 504A 5 - 117A 4 + 66A 3 + 30A 2 - 12A + 1 = 0. 

This polynomial also has a unique root in the interval (0,1), and the coordinates 
of the points z and <f>(,z) also satisfy inequalities which are similar to ©. Conse¬ 
quently on M + /g we have two six-link periodic trajectories which are invariant 
with respect to a permutation of order 3. □ 

Note that the polynomials Pz, Pq and Pr are reciprocal. This is because the 
described method allows one to find self-similar trajectories on M disregarding 
the condition Ao < 1. It remains to say that the mapping g(— 1) takes any 
self-similar trajectory on A4 to another self-similar trajectory, but it changes 
also the direction of the time flow. Hence Ao is replaced by -A. which also must 
be a root of the corresponding polynomial. 

2.8 Behaviour of optimal trajectories in the neighbour¬ 
hood of periodic trajectories 

We shall show that almost all optimal trajectories of problem m are attracted 
in the backward time direction to one of the two three-link cycles found in 
the preceding subsection, and in the forward time direction almost all optimal 
trajectories reach in finite time with a countable number of switches one of the 
three two-link cycles Z l3 . Here ’’almost all” means that on M/g there exists a 
set of points A' of full Lebesgue measure which satisfy this property. The cycles 
Q l and R ± do not belong to X. Moreover, there exists a countable number 
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of periodic trajectories that do not belong to X. The set ( M/g) \ X has a 
non-integer dimension, and the behaviour of the trajectories in it is chaotic. 

To obtain these results it is necessary to recall the results of m- In this 
subsection we shall describe the results of m along with the necessary expla¬ 
nations. 

We begin with the behaviour of optimal trajectories in the vicinity of the 
cycles Z ± and Zij. The cycles Z± that have been found in the preceding sub¬ 
section are repelling on the sphere M/g. More precisely, consider the Poincare 
mapping <I> of the switching surface to itself. Each of the cycles Z± intersects 
the switching surface at three points zf, i = 1,2,3. Each such intersection 
point is a fixed point of the mapping $ 3 . In the paper |17] the eigenvalues of 
the differential of the mapping $ 3 at the points zf have been calculated explic¬ 
itly. These eigenvalues, naturally, do not depend on the choice of the point zf 
and all their absolute values are strictly greater than 1. Hence in the vicinity of 
Z± any trajectory will approach the corresponding cycle Z ± in backward time. 
Let us remark that, generally speaking, this does not yet prove that almost all 
trajectories on M/g tend in backward time to Z ± . 

The behaviour of the optimal trajectories in the neighbourhood of the cycles 

has a different structure. As mentioned above, each two-link cycle consists 
of two parts Zy and Zy. The control on the interval Zy is non-singular and 
lies at the k vertex of the triangle fi, k ^ i,j. The control on the interval Zy is 
semi-singular relative to the edge ( ij ) and lies at its midpoint. 

Consider a point (xo,yo) £ M on Ay. i.e., Xo,yo £ Ay . Hence by Lemma 
Iwe have n(x 0 ,y 0 ) £ Zy. Suppose at first that the point (xo>2/o) is not a 
switching point and 7r(xo,2/o) lies on Zy. The control in 7r(xo,2/o) is the k 
vertex of the triangle f l,k ^ i,j. Then for a point (xi, y\) £ M sufficiently close 
to (xo,2/o)j the point E(x\ 1 y\) is sufficiently close to E(xo,yo) by Theorem|T| 
Consequently, the control on the optimal trajectory that emanates from (xi, y\) 
coincides during some interval of time with the non-singular control on Zy . 

Now suppose that 7r(xo,2/o) lies on Zy and, consequently, the optimal con¬ 
trol on Zy is the midpoint of the edge (ij) of the triangle ft. Denote (4> 0 , ip°) = 
E(xo,yo). Then if)® = -0° > i/>°. If the point (x\,y\) belongs to the neighbour¬ 
hood of (xo,yo)> (v'i'v) = E(x\,yi), and i/jj ^ ipj, then due to the continuity 
of the mapping E, the control on the optimal trajectory passing through the 
point (xi, j/i) must be either in the i or the j vertex of the triangle Cl. 

Moreover, in m it has been shown that there exists a neighbourhood V of 
the interval Zy in M/g with the following property. Let (xi, i/i) be an arbitrary 
point such that the image 7r(xi,2/i) lies in V. Then the optimal trajectory 
emanating from (xi,?/i) reaches •K~^[Zy] in finite time without quitting the 
preimage 7r _1 [H]. Therefore the optimal control experiences a countable number 
of switchings between the i and the j vertex of the triangle fb This process is 
similar to the chattering mode in the one-dimensional Fuller problem. The only 
difference is that the role of the interval of admissible controls is played by the 
edge (ij) of the triangle H. A detailed proof and exact analytic formulae can be 
found in [ T7j . 
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3 First theorem on the chaotic behaviour of tra¬ 
jectories in integral vortices 

In this section we prove Theorem [2] on the chaotic behaviour of trajectories in 
the integral vorte^f] of Hamiltonian system ([3]) in the case when the triangle 
Q in model problem © is sufficiently close to an equilateral one. This theo¬ 
rem is the first in a series of theorems on the chaotic nature of trajectories in 
Hamiltonian systems with discontinuous right-hand side which are proven in 
this paper. Moreover, the main results on the structure of the optimal synthe¬ 
sis in problem © obtained in this section also serve as a starting point in the 
proof that such chaotic behaviour of trajectories in integral vortices is generic 
for high-dimensional Hamiltonian systems with discontinuous right-hand side. 
For the case of an equilateral triangle we shall prove Theorem [G] in the subse¬ 
quent sections, which is a generalization of Theorem [2] and which among other 
results furnishes bounds on the Hausdorff and Minkowski dimensions of the 
non-wandering set and the corresponding topological entropy. 

3.1 Statement of the first theorem on the chaotic be¬ 
haviour of trajectories in the model problem 

Let us begin with the statement of the theorem on the chaotic nature of optimal 
trajectories in optimal control problem © and furnish all necessary explana¬ 
tions. The first three assertions of the theorem describe the set E C M+, which 
is the object of the theorem and is comprised of trajectories of Hamiltonian 
system ©• The last assertion describes the chaotic dynamics of the trajectories 
on this set. The set S is the analog of the set of non-wandering trajectories 
which is typical for integral vortices. 

Every trajectory on S intersects the stratified manifold S = S12 U1S13 U1S23, 
which represents the set of discontinuities of the right-hand side of system ©, 
a countable number of times. In Theorem [2] we therefore describe the chaotic 
dynamics of these trajectories in terms of the intersections of the strata Sij. 
More precisely, the sequence of intersections of these strata can be encoded by 
elements of the space £01 of bilaterally infinite words consisting of the letters 0 
and 1, equipped with the standard direct product topology. The space £01 is 
homeomorphic to Srnale Horseshoe. Denote by l : £01 —> £01 the topological 
Markov chain of the Bernoulli shift, i.e., let l be the mapping that shifts every 
word on position to the left. 

Theorem 2. There exists e > 0 such that if the angles of the triangle D in 
problem m are different from ^ by no more than £ and the distance from the 
of Q to the origin does not exceed ediamil, then there exists a subset £ 

7 By an integral vortex we mean the union of all trajectories of the dynamical system which 
pass through a given singular point. 

8 As the center of £7 one may take, e.g., the orthocenter, the centroid, the center of the 
incircle or the circumcenter. This is possible because the triangle is close to equilateral and 
the distances between these points do not exceed eCdiamQ for some fixed C > 0. 
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of the extended phase space M = T*M = {(a;, y, <f>, if)} of Hamiltonian system 
m with the following properties: 

(I) For every point z £ S there exists a time instant T(z) < oo such that the 
trajectory X(t,z) of Hamiltonian system fcff) which passes through z is 
well-defined and unique for all t £ [—oo,T(;s)]. Moreover, the trajectory 
X(t,z) hits the origin in time T(z), i.e., X(T(z),z) = 0. 

(II) The set 5 comprises trajectories of Hamiltonian system and is in¬ 
variant with respect to this system in the following sense. If z € S, then 
X(t , z)eH for all t £ [— oo, T(z)). 

(III) The projection of the trajectory Xft, z ) on the phase space M, prolonged 
by 0 for t > T(z), is optimal for all z £ S (i.e., S C M + ). The trajectory 
X(t,z ) intersects the switching surface S a countable, infinite number 
of times at time instants ... < f_i < to < t\ < t 2 ■ ■ ■ < T(z), i.e., 
X{tk,z) £ S, and where to < 0 < t±, and tk —> T(z) as k —»• +oo and 
tk — oo as k —> — oo. 

(IV) Consider the dynamical system defined by the map $:5n5->5ricS, 
which takes a point z £ S on S to the next intersection point of the 
trajectory X{t,z) with S, i.e., $( 2 ) = X(ti,z). There exists an integer 
n > 0, independent of the triangle Cl, such that the map is semi¬ 
conjugate to the topological Markov chain defined by the Bernoulli shift 

on the disjoint union of two copies of Smale Horseshoe. In other words, 

2 

there exists a surjective continuous map ’I'oi from Sfl<S to the space u ^01 
such that the following diagram commutes: 

2f 1 * 01 . 2 i 5,01 

US 01 ^USoi 

Here l denotes the left shift on each copy of £01 ■ 

3.2 Blowup of the singularity at the vertex of an integral 
vortex 

For the proof of Theorem [3 we need to perform a modified blowup of the origin. 
Form the topological viewpoint this means that we glue a sphere S 7 into the 
origin. For this construction Cl does not need to be an equilateral triangle. 
The set Cl can be an arbitrary convex compact set, the only condition is that 
0 £ Int Cl. i.e., the conditions of Theorem [1] have to be satisfied. 

Let us note, however, that the optimal trajectories of problem m reach the 
origin in finite time. Hence after the blowup the velocity vector field of ([31) will 
degenerate as we approach the sphere which was glued in. 
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Definition 4. The blowup of the singularity at the origin will be performed by 
a mapping 

B : (a;, y, <j>, ip) ^ (/z, x, y, <p, ip), 

where /i £ R+, and x £ R 2 , y £ R 2 , <p £ R 2 , and ip £ R 2 lie on the manifold 

Co = {|y| 24 + \x\ 12 +10| 8 + M 6 = 1 } c R 8 . (10) 

The mapping B is given by the formulae 

y = y/fj,, x = x/y 2 , <p = <p/p 3 and ip = ip / p 1 , (11) 

where 

M=(|l/| 24 + M 12 H^| 8 + IVf)^ (12) 

Let us carry over the action g of the group R+ to the cylinder given by the 
coordinates (/ i,x,y,<p,ip) by virtue of B, such that the map B is equivariant 
with respect to the action g, 


B o< 7 (A) = g{\) o B => g{\) (p, x, y,<p, ip) = (\p,x,y,cp,ip) VA > 0. 

The blowup of the singularity of system © at the origin has been performed 
by relations (I11I12D precisely because the action g of the group R+ can then be 
written in a very simple form. 

Definition 5. Let us denote the cylinder Co x {/r £ R} over Co by C. The 
manifold Co itself will be identified with the zero section, Co = C fl {y = 0}. The 
switching surfaces Sij shall be prolonged on Co in a natural way. By ’’vertical 
direction” we shall assume the direction of the tangent vector JC. 

Lemma 11. The blowup map B is a diffeomorphism from A4 \ 0 onto C D {/z > 
0}. 

Proof. From relations (11111211 it easily follows that the map B is well-defined on 
M \ 0 and maps A4 \ 0 bijectively onto C D {/z > 0}. Moreover, the map B is 
smooth on A4 \ 0. 

Let us show that the differential d B is non-degenerate on the sphere given 
by {M 24 + |cc| 12 + |(/>| 8 + \ip\ 6 = 1} = B -1 [{/r = 1}]. Regularity of dB at all 
other points will then follows from the equivariance of B with respect to the 
action g. The restriction of B on the spheroid B^ 4 [{/r = 1}] is by virtue of (1111) 
a diffeomorphism. Hence the restriction d~B\ Tz B- 1 [{/i=i}l of the differential is 
non-degenerate at every point z £ B -1 [{/r = 1}]. On the other hand, for 
z £ M. \ 0 we have 


d\ I A=i ^ S/ Jb(*) e ^ B ( z ) 


C. 
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If z £ B x [{y = 1}], then the tangent vector does not lie in the tangent 

space T(C D {/u = 1}). Hence the map d B \ z is surjective and therefore non¬ 
degenerate. □ 

Let us remark that the map B _1 is formally defined only on C fl {p, > 0}. 
However, we may define it on C fi {y < 0} by the same relations 


B 1 : (/. i, x, y, <p, ip) ( x , y, <p, ip), where y = yy, x = y 2 x, <p = y 3 (p, ip = y 4 ip. 

Then B 1 becomes a two-fold covering map over M. \ 0. 

If the set is a triangle, then a switching surface S is defined on the cylinder 
C. Namely, denote by S l:j the closure of the set of all points in C which are taken 
to Sij by the map B _1 . In other words, the set S i: j consists of those points 
(y,x,y,cp,ip) such that argmax ug Q (■;/>, u) contains the edge ( ij ) of the triangle 
Cl. Let us dehne also 5i23 = <Si 2 n<S 23 n«Si 3 and 5 = <S-L 2 UN 23 UN 13 . For ease of 
notation we will henceforth omit the tilde over Sij, S 123, and S, since this will 
never lead to any confusion. 

3.3 Reparametrization of time 

The discontinuous vector field on the right-hand side of ([3]) can be written as 
follows in the coordinates (n,x,y,<p,ip): 


A = T{x,y,<p,ip,u) = ±(24\y\ 22 (y,u) + 12\x\ w (x,y) + 8\cp\ 6 ((p,x) - 6\ip\ 4 (ip, (p)) 

x= jj{y~2Tx); 

y = 7i( u ~ T y)-’ 

(ip, u) —>• max. 

X ' u£f! 

(13) 


Here the triangle H does not need to be equilateral. 

The solutions of the ODEs m and © will be understood in the classical 
sense of Filippov (see (2j) as solutions of a system of ODEs with discontinuous 
right-hand side. 

Denote the vector field C D {y > 0} on the right-hand side of (fl3l) by £. 
Formally the vector field £ is defined for y > 0, but we prolong it by the same 
formulas onto the lower half C 0 {y < 0} of the cylinder. Then B _1 will take 
the vector field £ to the vector field of system © for y, > 0 as well as for y < 0. 

Note that for y —> 0 the field £ grows as jj ■ However, the field y£ can already 
be prolonged onto the zero section Co = Cfi{^ = 0} of the cylinder C at all points 
where the covector ip is not orthogonal to any of the edges of the triangle D. 
The integral curves of the vector field y£ either do not intersect the zero section 
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Co or are contained in it. This is because we have /i = /iY along the vector field 
/i£, where the vector field Y does not depend on \i. Moreover, the components 
of the vector field yf which correspond to the coordinates (x, y , <f> , ip) do not 
depend on y. Hence every integral curve of the vector field yf which lies in Co 
can be lifted in a unique manner to C n {y 7 ^ 0} if the initial value of y is given. 
On the other hand, every trajectory lying in Cn{y 7 ^ 0} projects to a trajectory 
in Ca¬ 
dence the integral curves of the vector fields £ and yf coincide onCfljc 7 ^ 0 }, 
but the velocity on the trajectories is different. Let us denote the time parameter 
for the movement on the trajectories of the vector field yf by s. Then the 
parameters s and t are related by 


ds = —dt. 

M 

The time parameter s allows to characterize the optimal trajectories by 
their limiting behaviour as they approach Co and is hence of advantage in the 
subsequent considerations. This behaviour is described in the following two 
lemmas. In the first lemma we prove that the function y(s) decays exponentially 
on any optimal trajectory, and in the second lemma we give a sufficient condition 
of optimality of a trajectory in terms of the decay rate of the quantity y(s) on 
this trajectory. 

Lemma 12. Consider the image of an optimal trajectory 
(%{t, q 0 ), y(t , q 0 ), (pit, q 0 ),ip(t , q 0 )) 

on C fl {y > 0} for t < T(qo). Fix time instants to = t(so) < T(qo) and 
t\ = t(s 1 ) < T(qo). Then there exist positive constants 71 and 72 , independent 
of the trajectory and of the choice of to and t\, such that 

L»iAi 0 e“ 7l(si “ So) < Mi < L> 2 Moe“ 72(si " So) , 
where y k = p(x(t k ,q 0 ),y(t k ,qo),<j>{t k ,qo),ip(tk,qo)), k = 0,1, and D 1 = = 

72 
71 * 

Proof. Denote short-hand 

p(t) = n(x(t,q 0 ),y(t,q 0 ),<j>(t,qo),ip(t,qo)). 

The bounds obtained in Theorem [T] immediately yield the existence of 71 > 0 
and 72 > 0 such that for every initial point go we have 


72 (Y(g 0 ) - t) < y(t) < 71 (T(q 0 ) - t) 
on the optimal trajectory emanating from go- Hence 


si — So 


tl 



to 


< J_ In —*o . 

- 72 T(qo)—ti > 


> 


■In 


T(go) — t 0 


— 71 T (qo)-ti ' 


(14) 
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It follows that 


71O1—«o) < Tjqo) ti 72 ( Sl _ So ) 

- T(q 0 ) -t 0 ~ 6 

The proof is concluded by application of bounds (HI. □ 

Corollary 8. On any optimal trajectory we have p —> +0 and s —> +oo as 
t —> T(qo ) — 0. 

Lemma 13. Let z(s), s € R, be a trajectory of the vector field /i£ on the upper 
half C fl {p > 0} of the cylinder C. If /i(s) —» 0 as s —> +oo and 

r+oo 

T= p(s)ds < oo, 

J o 

then on the trajectory z(s) we have s(t) —> +oo as t —> T — 0 and s(t) —> —oo 
as t —► — oo. Moreover, the trajectory B 1 (z(s(t))) lies in M + and is optimal, 
given we prolong it by zero for t >T. 

Proof. Since dt = /ads, we have 


t= p(a)da 

Jo 

on the trajectory z(s). By positivity of /z(s) we have that t —> T — 0 if and only 
if s —> Too. 

Now consider the trajectory z(t) = B _1 (z(s(f))), which is defined for t < T. 
By virtue of (fl2l) we get x, y, (j), ip —> 0 as t —> T — 0. Since the trajectory z(t) 
is a trajectory of Hamiltonian system © we have by virtue of Corollary [7] that 
z(t) is optimal if prolonged by zero for t >T. 

The time when the origin is reached by an optimal trajectory can by Theorem 
[I] Ire bounded from above by a function of the distance to the origin. Hence there 
exists a neighbourhood of 0 in M such that the trajectory z(t) stays outside 
of this neighbourhood for all t < 0. Hence the function ji(s) is bounded away 
from zero for s < 0. It follows that f Q °° p(s)ds = —oo, i.e., t(s) —> —oo as 
s —>■ — oo. □ 

Remark 6 . Note that the section Co can be identified with the quotient space 
(M. \ 0)/g, and the trajectories of the vector field can be interpreted as 
images of the trajectories of system m under the natural projection (A 4 \ 0) —r 
(M \ 0 )/g. However, the concrete representation of (At \ 0 )/g as the section 
Co has some advantages. It yields a system of ODEs, defined by the vector 
field pf, on the whole cylinder C, it yields the uniform convergence of optimal 
trajectories to Co, and it open the possibility to define optimality by virtue of 
the preceding lemmas. 
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3.4 Robustness of the self-similar trajectories 

In this subsection we investigate what happens to Hamiltonian system © if the 
triangle 17 is replaced by a nearby triangle 17'. 

Let the triangle 17' be such that each vertex of 17' is located in an e- 
neighbourhood of the corresponding vertex of 17, where e > 0 is sufficiently 
small. Then 0 G Int 17' and 17' fulfills the requirements of problem (JT]) . All ob¬ 
jects which were introduced above and which are related to the triangle 17' will 
be marked by a prime. E.g., the vector field 031 corresponding to the triangle 
17' will be denoted by £'. 

Since for all i ^ j the edges ( ij ) and ( ij )' of the triangles 17 and 17' are 
close to each other, the switching hyperplanes S, 3 and SC will be situated at an 
angle ay in extended phase space M which is close to zero. In order to define 
closeness of mappings defined on S l3 and SC in the sequel, we will identify the 
hyperplanes S, 3 and SC by virtue of the map pr given by 

pr(z, y , </>, VO = (x, y, (/>, O aij ip) G S' i:j , where (x, y , <j>, if) G <Sy, 

and O aij G 0(2, R) denotes a rotation of the plane by the angle ay. The 
map pr commutes with the action g of the group R + and can be continued to 
the intersection <Sy n Co of the switching hyperplane with the zero section in a 
smooth manner. 

Lemma 14. Consider a trajectory on Co of the vector field gf for the triangle 
17. Suppose the trajectory intersects the switching surfaces Sij fl Cq and Sjk D Co 
transversal 1$^ at points A and B, respectively , and assume A, B ^ £123 . Here 
we may have k = i, but we assume j ^ i,k. Then for all r G N and a > 
0 there exists e > 0 such that if each vertex of some triangle 17' is located 
in the e-neighbourhood of the corresponding vertex of the triangle 17, then the 
Poincare return maps >I> and <E ,/ defined by the vector fields gf and gff are en¬ 
close diffeomorphisms of neighbourhoods of A and A' = pr A in the C r metric. 

Proof. In order to work in some neighbourhoods of Sij and Sjk we have to 
extend the vector field gl; in a smooth manner onto these neighbourhoods. This 
can be accomplished by relaxing the last condition in (1131) and setting the control 
equal to the vertex j of 17 throughout the neighbourhoods. We shall denote the 
resulting velocity vector field by gf. Performing a similar operation with the 
system corresponding to the triangle £7' we obtain a vector field gff. 

Since the switching surfaces are closed and the vector field gf is transversal 
to and Sjk at A and B 1 we have that every trajectory which emanates from 
a point on <Sy sufficiently close to A will intersect S in the vicinity of B for 
the first time. Hence the Poincare return map will not change if we replace 
the vector field gf by the vector field gf. The relevant arcs of the trajectories, 
which come out of a neighbourhood of A on Sij and lead to a neighbourhood of 

9 The jump of the vector field //£ at the point A (B) on the hypersurface 5y (Sjk) is 
tangent to this hypersurface. Therefore it is irrelevant for the definition of the transversality 
of the intersection from which side we take the limit of the velocity vector as we approach the 
switching surface. 
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B on Sjk without control switchings, are left unchanged. A similar statement 
holds true if the vector field jj£f is replaced by the vector field p,f. 

Since the vector fields pf and p£f are smooth, the assertion of the lemma 
follows immediately from the transversality of the vector field pf to the switching 
surfaces Sjj and Sjk at the points A and B , respectively. □ 

Each trajectory z(t) of system ([3]) can be put in correspondence to its image 
n(z(t)) on Cq = JA/g under the canonical projection tt : M —> M/g. This 
image can be found explicitly as follows. We send the trajectory z(t) to the 
cylinder C by virtue of the map B and discard the coordinate /i. Under this 
operation the self-similar trajectories transform to periodic trajectories of the 
vector field pf on Co¬ 
in this subsection we study the robust periodic trajectories on Co- We shall 
apply the obtained results to the six-link cycles R± which have been found in 
Lemma [TUI 

Definition 6. We shall call an optimal self-similar trajectory z(t) of system (HI)) 
robust if it is not semi-singular, does not intersect £123, and its image 7r (z(t)) 
on Co is a robust periodic trajectory. Here the latter means that (i) Tr(z(t)) 
consists of a finite number of smooth pieces (links), and (ii) the differential 
of the Poincare return map at the switching points of ir(z(t)) does not have 
eigenvalues equal to 1, apart from the trivial eigenvalue 1 corresponding to the 
velocity vector along the trajectory. 

The definition above is well-defined, in the sense that the Poincare return 
map along such a periodic trajectory is a smooth diffeomorphism by virtue of 
Lemmas [9] and 1141 It turns out that the robust self-similar optimal trajectories 
are preserved under small changes of the triangle f2. 

Lemma 15. Let z(t) £ A 4 be a robust optimal self-similar trajectory of problem 
m■ Then for all r gN and a > 0 there exists a number e > 0 such that if every 
vertex of some triangle f V is located in the e -neighbourhood of the corresponding 
vertex of the triangle H, then in problem m defined for the Biangle f V there 
exists a robust optimal self-similar trajectory z'(t) with the following properties. 
The periodic trajectories tt( z(t)) and tt( z'(t)) are a-close in the C°-metric, and 
the Poincare return maps on the cylinder C defined for systems m with triangles 
Q and ft' are a-close in the C r -metric in the neighbourhood of the switching 
points of the trajectories tt( z(t)) and n(z'(t)), respectively. 

Proof. Consider the periodic trajectory z(s) of the vector field pf that is ob¬ 
tained as the vertical projection of the image B(2(f)) on Co- Let so be a time in¬ 
stant such that z(s) is smooth in a neighbourhood of s = s 0, and let L E3 z(so) be 
a small smooth piece of hypersurface of codimension 1 in C which is transversal 
to z(s). Since z(s) £ T%i s \Co, we have that the hypersurface L is also transversal 
to Co- Denote by $ the Poincare return map <f> : L —>■ L along the trajectories 
of the vector field pf. Then the point z(so) is a fixed point of the map <f>. 

Let now si < ... < s m be the points of non-smoothness (the switching 
points) on the trajectory z(s), labeled in consecutive manner and such that si 


33 



Figure 2: Schematic representation of the results of Lemma fl6l 


is the first point of non-smoothness after so- Denote by Li the intersection of a 
small 6 -neighbourhood of the point z(si) with the corresponding hypersurface of 
discontinuity Sij of the vector field /r£. Then the Poincare return map <f> : L —>■ L 
can be written as a composition of the maps 


$ : L 


L2 


— 1 


->L r , 


It follows by virtue of Lemma [14] that $ is a smooth diffeomorpliism. 

Let $ be the restriction of $ on Co- By virtue of the definition of robustness 
of the self-similar trajectory z(t) the differential is robust at the point z(sq), 
i.e., it has no non-trivial eigenvalues equal to 1. The map 4> can be expressed 
explicitly in terms of $ by 


$0, x, y, 0, 0) = (A on, $(x, y, 0,0)), (15) 

where Ao comes from the definition [2] of self-similarity. Hence the differential 
d$U (so) contains the additional eigenvalue Ao as compared to d4>| z ( So \. By 
optimality of z(t) we have 0 < Ao < 1 (see Remark 0. Hence d4>| z ( So ) fulfills 
the conditions of robustness. 

Let us now describe how the map $ changes if the triangle Cl is replaced 
by a close-by triangle f2'. Consider the vector field yt;' on C which is obtained 
from system (0> with the triangle Cl'. Since the triangles Cl and Cl' are close, we 
immediately obtain that the maps and 40 are close in the C^-metric. Hence 
also the maps 4> and are close. Since the point z(so) £ L is a robust fixed 
point of the map 4>, it will be preserved by a small perturbation of $ in the 
C 1 -metric (see e.g., [5], Proposition 1.1.4). □ 

Let us finally note that if the triangle Cl is contracted homothetically by 
a factor of A > 0, then the trajectories of system (0) will be left unchanged. 
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What changes is the velocity on them, which will change by a factor of A. A 
rotation of the triangle f2 about the origin leads to a rotation of the whole 
optimal synthesis by the same angle. Hence the assertions of the preceding 
lemma can be reformulated in terms of closeness of the angles and the centers 
of the triangles fl and fl', as it was done for the equilateral triangle in the 
formulation of Theorem [2] 

3.5 Homoclinic orbit on the zero section of the cylinder C 

In this subsection we construct a homoclinic orbit of the blown-up system m 
for the case of an equilateral triangle 11 and show that it is stable under small 
perturbations of the triangle 12. 

Let hence 11 be an equilateral triangle with center in the origin. In the paper 
HZ! it has been shown that the self-similar trajectories Z ± , Q l , and R ± from 
Lemma [TUI are robust (see [17] . Propositions 6.3, 6.4, and 6.5). Moreover, it has 
been shown that the periodic orbit Z ± is repelling (more precisely, all nontrivial 
eigenvalues of the differential d<f> are real and have modulus strictly greater than 
1), and the periodic orbits Q l and R ± are hyperbolic (more precisely, exactly one 
non-trivial eigenvalue of d$> has modulus strictly smaller than 1, all others have 
modulus strictly greater than 1). Hence if the triangle fl is close to equilateral 
and its center is close to the origin, then by virtue of Lemma [15] there still exist 
periodic orbits of system ([3]) on Co which are close to the orbits Z ± , Q l , and 
R ± . We shall keep the notations for these orbits. The Poincare return maps 
along these periodic orbits are also close to those for the case of an equilateral 
triangle. 

It follows that on 5 fl Co there exists a 1-dinrensional smooth stable manifold 
and a 5-dinrensional smooth unstable manifold of the Poincare return map d* : 
S —> S in the neighbourhood of each point of the intersections Q l fl S and 
R^ fl S. Indeed, by virtue of Lemma [14] the map $ : S —> S is smooth in 
the neighbourhood of the intersections Q l fl S and R± fl S. But in general 
the map $ is discontinuous on S. Moreover, if we extend the 1-dinrensional 
stable manifold further away from the intersections Q l J~1 S and R ± fl S, it will 
stringently encounter the discontinuity set of the map $. Nevertheless, we have 
the following lemma. 

Lemma 16. Assume that the triangle fl is sufficiently close to an equilateral 
triangle with center at the origin. Then there exists a homoclinic point z 0 £ Co 
on the switching surface S such that the iterations 4> n (z 0 ) tend to the six-link 
periodic orbit R + flS as n —► ±oo (see Fig. \3). Moreover, for every sufficiently 
small e-neighbourhood V C Co of R + fl S there exists a 5-neighbourhood W C 
C 0 n<S of Zo and integers m > 0 and l < 0 such that (i) the images and 

&(W) are contained in one connection component ofV; (ii) the restrictions of 
the maps 4> m and to W are diffeomorphisms; and (in) the image under the 
map of the stable manifold of the periodic orbit R + C\S in the $ m (W) and 
the image under the map <I> _i of the unstable manifold in 4 >i (IT) intersect at the 
homoclinic point zq, and this intersection is transversal. 
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A similar statement holds for the periodic orbit R HiS. 

Proof. It is sufficient to prove the existence of a transversal homoclinic point 
zq for the case of an equilateral triangle Q and to insure that the trajectory 
on which z o is situated is bounded away from the set £123 of discontinuity of 
the Poincare return map. Both conditions have been verified by a numerical 
simulation of system m for the case of an equilateral triangle, as we shall 
detail below. 

Denote the 5-dimensional unstable manifold of the map d> 6 in the neighbour¬ 
hood of some switching point S(so) on the periodic orbit R + by H 5 C CoflS. Let 
H® be the connection component of the set H$ \ £123 which contains the point 
z(so)- Then the restriction <f> -6 | ff o is smooth, 6 [^ 5 ] C H 5 , and z(sq) € H ° 
is the unique fixed point and attractor of the map <E>~ 6 . 

Let do, <Ti,..., uq = (To be the 1-dimensional stable manifolds of the map <f> 6 
emanating from the points .z(so),-?(si), • • • ,5(se) = z(sq) of the periodic orbit 
R + , respectively. Note that the map d) takes the curve Ofc to the curve <Jk+i and 
simultaneously contracts the image curve towards the switching point z(sk+ 1). 
The set of trajectories of system (fl3l) which pass through the curves ak form a 
cylindrical 2-dimensional surface E, which contains the periodic trajectory R + . 
The curves au are a subset of the intersection E fi S. 

Let us consider the behaviour of the curve ak when it encounters the discon¬ 
tinuity hypersurface £123 C S of the map d>. Recall that £123 is the intersection 
of the three strata £12, S13, S23 °f the switching surface S. Assume that the 
trajectories of system (fTSl) intersect the switching surface S transversally in the 
neighbourhood of some point z € <Si23- Then these trajectories experience one 
switching on one side with respect to £123 and two consecutive switchings on 
the other side. This situation is schematically depicted in Fig. [3a] Hence the 
curve (Tfc either splits into two branches or two branches merge into one for each 
encounter of £123. The resulting net of curves still lies on the cylindrical surface 
E and equals the intersection E D S. Let us remark that the preimages of the 
branching points under the map are points of non-smoothness of the curves 
making up the set E fl <S. 

If we track the curves at in the direction pointing away from the switching 
points z(sk) of the periodic orbit R + , then we see that the curves <r; c with 
odd index k pass through a branching point, and the curves <r*. with even 
index k encounter a point of non-smoothness, which is the preimage of the 
above-mentioned branching point® After passage through the point of non¬ 
smoothness the curve (To transversally intersects the unstable manifold H® , thus 
defining the sought homoclinic point zq. This situation is schmatically depicted 
in Fig.l3bl On this figure we see the cylindrical surface E. In order to obtain the 
cylindrical topology, one should identify the top and the bottom of the picture. 
The hatchings encode the control used in system (El), taking values in the three 

10 Whether the curves with odd indices encounter branching points and the curves with 
even indices non-smoothness points or vice versa depends of course on the indexation of the 
switching points on R+ and has no intrinsic meaning. The point is that the branching points 
and the points of non-smoothness alternate. 
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(a) Behaviour of trajectories in th 
vicinity of <Si 23 


(b) Cylindrical surface E, curves Ok, 
homoclinic point z o and trajectory of 
system m passing through zo 


Figure 3: Schematic representation of the curves <7*,. 


vertices of the equilateral triangle Q. The periodic orbit i? + is represented by 
the bold vertical line, the curves Uk by the horizontal semi-bold lines. The thin 
line represents the trajectory passing through the point zq, the dashed lines 
symbolize the identification of the intersection points of this trajectory with the 
curves erg and erg, respectively. The periodicity of the whole picture follows from 
the invariance of the periodic orbit R + with respect to a permutation of order 
3 of the symmetry group S 3 . 

From the above it follows that we can set the numbers m, l, e.g., to m = 7 
and l = 0. Moreover, it is clear that the trajectory passing through zg is bounded 
away from the discontinuity surface £123 ■ 

The proof for the periodic orbit R~ follows similar lines. 

□ 

Let us remark that the neighbourhood W in Lemma [16] can be replaced by 
an arbitrarily small neighbourhood that is contained in W. 

3.6 Conclusion of the proof of the first theorem on the 
chaotic behaviour 

It is well-known that a homoclinic point generates a Smale Horseshoe in the 
neighbourhood of a periodic point. In our case the Poincare return map is for¬ 
mally non-smooth, but Lemma [16] allows to ignore the discontinuities of the 
return map and to directly use the classical theorem stating that a transversal 
homoclinic point generates a Smale Horseshoe in an arbitrarily small neighbour¬ 
hood of the periodic point. 

Proof of Theorem [H We shall start with the construction of the set 5. For this 
we need to lift the Smale Horseshoe in the neighbourhood of the 6-point periodic 
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orbit R+ flS of the map $ from the zero section Co to the whole cylinder C and 
to show that all trajectories in this lift tend to Co and satisfy the conditions of 
Lemma 1131 

Consider the map < f> 6 , defined on S C C in the neighbourhood of the points 
of R+ n S. Since the cycle R + is obtained from a self-similar trajectory with 
Ao < 1, we get by virtue of (fl5l) that d$ 6 (-J^) = \ 0 -J^ at the points of R + n S. 
Let us fix one of the six points of R+ n S and consider its neighbourhood in 
which the map <f> 6 is contracting in the vertical direction. This neighbourhood 
can be chosen as a cylinder over some neighbourhood in Co, since by virtue of 
m the contraction condition does not depend on /r. 

Let hence Vo C V be a neighbourhood of one of the six points of R + fl<S in Co 
such that the map <f> 6 is contracting by a factor of Ao < 1 on Vp x {/* g 1} in the 
vertical direction. By virtue of Lemma HUl on Vo there exists a Smale Horseshoe 
A 0 for some iteration ($ 6 ) 7V< (see, e.g., [3], Theorem 6.5.5). Set N = 6 N'. 

The Smale Horseshoe A 0 is partitioned in two subsets as follows. Define two 
subsets Wq and W\ of the neighbourhood V. The set Wo is defined as $ ; (W), 
where the neighbourhood W 9 Zo and the power l are taken from Lemma [16] 
(see Fig. [2]). Hence for every point 2 £ Wo the relation $ m ~ l (z) £ V holds, 
and the map & m ~ l \w 0 is a diffeomorphism. The set Wi consists of those points 
z £ V which do not leave this neighbourhood of i?+n<S under the iterations $ fc , 
0 < k < m — l , i.e., Wi = fYk=o The map ^ m ~ l \w 1 is a diffeomorphism. 

Clearly Wo D Wi = 0. Hence the Smale Horseshoe A 0 consists of two disjoint 
subsets, A 0 = (A 0 D Wo) U (A 0 D Wi). 

Let z £ (Wo U Wi) (~l 3> _Ar (Wo U Wi). Consider the trajectory of the vector 
field /x£, i.e., of system m , emanating from z. Denote by S(z) the time (in the 
parametrization by s) which the trajectory needs in order to cross N consecutive 
times the strata Sij of the switching surface. Since the restrictions of the map 
<F m ~ i to Wo and Wi are diffeomorpliisms and the trajectories of the vector field 

in (WoU Wi)n$ _Ar (WoU Wi) intersect the switching surface S transversally, 
we have the bounds 


0 < Sjnin < S{z) < S'max (16) 

for some constants <5jni n and Sjnax- 

Note that for every point of the Smale Horseshoe Ao the trajectory of the 
vector field through this point exists and is unique for all s £ (—oo,+oo). 
Indeed, every power $" of the Poincare return map is well-defined on the points 
of A 0 , since A 0 C Vo fl 3> -Ar (Vo) and >I >7V (Ao) = A 0 . Moreover, the trajectory 
of the vector field ^ emanating from an arbitrary point of Ao intersects the 
switching surface only transversally and is hence unique. Its existence for all 
s £ (—00, +00) follows from the bounds (fTBl) on the transition times. 

To every point z £ Ao of the Smale Horseshoe we put in correspondence an 
infinite sequence ’F( 2 ) £ E 0 i consisting of 0 and 1. We define the position j £ Z 
of ’I'(.z) to be equal to 0 if all iterations <f> fc (z) for k between jN and (j + 1 )N lie 
in Vo, and we set it equal to 1 if (ar) is in the neighbourhood of the homoclinic 
point zo for some k £ (jN, (J + 1 )N). Note that the iterations & N (z) lie in Vo 
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in any case. 

The restriction of <!> v onto the cylinder over Ao x {/i € R} in C is in gen¬ 
eral not contracting in the vertical direction, regardless of the fact that <f> 6 is 
contracting in the neighbourhood of i?+ D S. This happens because during the 
N iterations of the map $ the points of the Srnale Horseshoe Ao may leave the 
neighbourhood of i?+ D S and visit the neighbourhood of the homoclinic point 
z 0 - 

Let A max be the maximal factor of expansion of the map <& N in the vertical 
direction for the points in the cylinder over Aq, 


A 


max 


max 

zeA 0 xR\0 


fJ,($ N (z)) 

n{z) 


The maximum is well-defined, because Ao is compact, and the continuous func¬ 
tion does not change if we replace 2 by g(X)z for any A 7^ 0. 

It follows that if the position j of the sequence 11/(2) equals 0, then the map 
<1> v is contracting in the vertical direction by a factor of Ain the neigh¬ 
bourhood of the point & N (z), because the iterations do not leave the 

neighbourhood of R+ fl S for k £ [jN, (j + 1)-/V]. However, if the position j of 
the sequence ^'(2) equals 1, then the map $ N extends the vertical direction by 
a factor of at most A max in the neighbourhood of & N (z). 

Let us choose K € N, K > 2 such that A ( ^' V ' /fJ A max < 1. Consider the Smale 
Horseshoe Ai C A 0 consisting of those points 2 £ A 0 for which the sequence 
\I/(2) has zeros at all position indices which are not divisible by K. At the 
positions whose indices are divisible by K we can still have 0 as well as 1. Then 
the restriction of the map <& KN on Aj x {/* 6 R} contracts the vertical direction 
by a factor of at least Ai = A ( j S ;V ^'A max < 1. Set the integer n from item (IV) 
of Theorem [2] equal to KN. 

Define the set Si as follows. Consider the upper half of the cylinder Ai x R + 
and pass a trajectory of the vector field yt; through every point of the obtained 
set for s £ (—00, +00). Transport the obtained set to the original extended 
phase space Ai by virtue of the map B _1 . 

In the neighbourhood of the trajectory I?_ there also exists a Smale Horse¬ 
shoe A2 for the map <h n . Without restriction of generality we can take equal 
powers of the map <f> for the Horseshoes Ai and A2. Define a set S2 in a similar 
way as we defined Hi- These two sets are disjoint. Indeed, for every point z £ 
the majority of the iterations $*(2) lie in the neighbourhood of R+ (~I<S, namely 
at least (K — 1 )N out of every KN , where K > 2. A similar statement holds for 
the Horseshoe A2 in the neighbourhood of R- fl S, and we immediately obtain 

Si ns 2 = 0. 

Define S = Si U S2. 

The time T(y) in item (I) of Theorem [2] exists by virtue of Lemma 1131 In¬ 
deed, the map $" : Ai x {y > 0} —> K\ x {y > 0} contracts the coordinate n(z) 

~(K-l)N /6 

of every point 2 £ Ai x {/z > 0} by a factor of at least Ai = Ao A max < 1. 

The parameter s on the trajectory X(z,s(t)) grows by an amount of KS{z) £ 
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[KS m i n , iTS'max]. Hence the parameter p(z(s )) exponentially decays on the tra¬ 
jectory emanating from 2. Therefore the conditions of Lemma [T3l are satisfied. 
Uniqueness follows from the definition of the Poincare return map. 

Items (II) and (III) of Theorem [3] follow from the construction of the set 5, 
optimality also follows from Lemma |13l In order to prove item (IV) we define 
the sequence ^01(2) G £01 for z G Hi as the subsequence of \t(7r(,z)) consisting 
of the elements at positions with indices divisible by K, H/01 (z)j = i b(z)jK for 
all j e Z. For 2 G S 2 the sequence ^01(2) lies in the second copy of the Smale 
Horseshoe £01 and is defined in a similar way. □ 

Theorem [5] proven above allows to find the elements of the dynamical system 
defined by the Bernoulli shift l : £01 -3► £01 with positive entropy in the original 
model problem ©■ For instance, we have the following corollary. 

Corollary 9. If the triangle fl satisfies the conditions of Theorem [H then there 
exists a countable, infinite number of different 1-parametric families of self¬ 
similar trajectories in the optimal synthesis of problem W- 

Proof. Consider an arbitrary periodic trajectory of the Bernoulli shift l. Clearly 
there exists an infinite number of such trajectories. The preimage of an arbitrary 
such trajectory under the mapping ’F01 yields a 1-parametric (with respect to 
the action g of the group M+) family of self-similar trajectories. □ 

4 Topological properties of the Poincare return 
map 

In Theorem [2] we obtained a semi-local result on the existence of chaos in the 
optimal synthesis of problem © with an arbitrary triangle fl, given it is suffi¬ 
ciently close to an equilateral triangle centered at the origin. For the equilateral 
triangle the optimal synthesis can be described exactly. In Theorem [G] we de¬ 
scribe the non-wandering set of trajectories and the graph of the topological 
Markov chain conjugated with the Poincare return map <f>. We find bounds on 
the Hausdorff and Minkowski dimensions of the non-wandering set of trajecto¬ 
ries and the corresponding topological entropy. In order to obtain these results, 
we start with the consideration of the global topological properties of the return 
map $ of the switching surface S on itself. 

In this section we assume that the triangle fl is equilateral and centered on 
the origin, if not we state it explicitly. 

4.1 Topological structure of the switching surface 

The investigation of the structure of the optimal synthesis in the model problem 
© with equilateral triangle U is based on a thorough study of the Poincare 
return map <F. As was stated earlier, the map $ is defined on some subset 
of the switching surface. In this subsection we first give an exact topological 
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Figure 4: Schematic representation of the disc D t j of the switching surface 7r(«S). 


description of the switching surface itself and of the subset of points on which 
the map <f> is defined. 

The switching surface S can be described by means of the variables ipi- 
Hence the intersection M + n S is most easily described in the space of adjoint 
variables N = {(<p,ip)}. We shall identify this space with M and M + by virtue 
of the map E. Let e : N/g —> N be a right inverse of the canonical projection 
7r : iV \ 0 —^ N/g. Let us for convenience define the embedding e by 

(e ° tt) (</>, 4’) = V)> where g = Vl</>| 4 + IV’I 3 - 

We have chosen this way of defining e because then e commutes with <?, and 
the projection 7r can be imagined as mapping the point (bo, 4o) S N \ 0 to 
the intersection of its orbit g(M + )(0Oi 4 > o) with the spheroid e(N/g) = : 

|</>| 4 + |-i/’| 3 = 1}. The space N/g is endowed with the structure of a smooth 
Riemannian manifold, the map 7r becomes a smooth surjection, and e is a smooth 
embedding. 

The intersection S D M + consists of three 2-dimensional half-spheres D &, 
defined by the relations -!/>, = i[>j < if>k ■ These half-spheres intersect each other 
in the circle given by i/> = 0, 

D k = e{N/g) n {(0, ip) £ N : 4>i = tpj < ip k } 

and 


D i D D 2 n D 3 = e(N/g) D {(4>, ip) & N : ip = 0}. 

Each half-sphere D k is in turn divided into two discs, D k = Dij U by the 
plane (pi = where 
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Dij = e(N/g) D {(<p, ip) £ N : ipi = ipj > ip k and (pi < cpj}. 

Define also 

d k — D Dji — D k Pi { (pi — 0 j }. 

On every optimal trajectory which passes through a point in Dij\d k the control 
switches from the vertex i of the triangle fl to the vertex j. 

The circle e(N/g) fl {ip = 0} is divided into 6 arcs (see Fig. [4j) 

d° ijk = e{N/g) D {((p, ip) : ip = 0 and fa < (pj < cp k }. 

The arcs are separated by the points 


dl = e(N/g) D {(p k > (pi = (pj} and d k = e(N/g) 0 {(p k < (pi = (pj}- 

Thus the image 7 r(S) of the switching surface on the quotient space N/g is a 
2-dimensional finite CW-complex with cells D^, df- k , d k , and d k . It is not hard 
to see that the surface 7r(<S) is homotopically equivalent to the wedge sum of two 
2-dimensional spheres. Let us choose orientations on the arcs d^ k as a function 
of the parity of the permutation ( ijk ) as is shown on Fig. [I] The orientations 
will be useful when factoring with respect to the discrete symmetry group S 3 . 

The Poincare return map $ is then defined at all points of the image 7r(«S) 
except those on d k . 

Since the return map $ commutes with the action of the group S 3 , we may 
restrict our consideration to one of the discs Dij. The action of S 3 on 7r(<S) 
respects the structure of the cell complex and can be described as follows: 

1. The group S 3 acts freely and transitively on the 0-dinrensional cells df 
and dp . 

2. The group S 3 acts freely and transitively on the 1-dimensional cells d ( - jk 
and preserves their orientation. 

3. A normal subgroup of S 3 acts freely and transitively on the 1-dimensional 
cells d k - The stabilizer of the cell d k is the subgroup generated by the 
transposition (ij) £ S 3 . 

4. The group S 3 acts freely and transitively on the 2-dimensional cells Dij. 

4.2 Factorization with respect to the action of the group 

S3 

Before we start with the description of the properties of the map let us 
first introduce some conventions which will facilitate the text and make it more 
comprehensible. The image 4>(z) of an arbitrary point 2 £ Dij lies either in 
Dj k or in Dji. By application of the corresponding permutation cr £ S 3 we 
may return it to the surface Dij, (a o < t>)(z) £ Dij. In this way the trajectory 
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{$ ra (~)} ngZ of an arbitrary point z £ Dij can be represented on D t j only. In 
this section, if we say that the image of a point z £ Dij under the map $ is some 
point z' £ , we always mean that the corresponding permutation a £ S3 has 

been applied. By some abuse of notation we shall nonetheless write $(2) = z'. 

We could also pass directly to the quotient space S/S 3 . The Poincare return 
map <f> can be correctly carried over to S/ S 3 , since it commutes with the action 
of S 3 . However, the way described above, albeit equivalent, seems preferable to 
us, because it leaves the topology of the disc Dij unchanged. 

We shall perform all computations in the quotient space N/g, identifying it 
with M/g and M + /g by virtue of the map E according to Theorem[T] Then the 
disc D^ C {N/g) can be seen as a subset of N, given explicitly by the system 
of inequalities 


' I0I 2 + I0I 2 = i; 

tpi = ipj < 4>k', 

* 0i — dj ; 

+ 02 + ^3 = 0; 

01 + 02 + 03 = 0. 

Note that in the preceding subsection we used the normalization |0| 3 + |0| 4 = 
1 which has the advantage that it is compatible with the action g of the group 
K.+ . But it will be more convenient for us to perform the numerical calculations 
with the normalization |0| 2 + |0| 2 = 1. 

On the disc D^ we introduce the coordinates 

j Xi = i(0; - 0j) - \/30i, 

\ X 2 = \[i<t>k. 

Then the disc D^ is mapped to the oblate disc 2y 2 + X 2 ^ 1; an d the 
variables 0 and 0 can be recovered from the coordinates \i , X 2 by the formulas 


0i 


0i 

0fe 


i(- v / 2 / 3 x 2 -X 1 + V 1 - xi - xi); 
\ (-1/273x2 - xi + V 1 - xi - xi); 
\/ix2; 


0i = 

0j = 

0fe = 


Xl + 01-xj-xl 
2 V 3 

Xl+\/l-xf-x| . 
2 V 3 ’ 

Xl + x/l —X?—xi 


4.3 Known pieces of the synthesis 

For definiteness, we shall consider the map $ as acting on the disc D31. In 
this subsection we describe the subsets of D31 which correspond to pieces of the 
optimal synthesis described in the paper m- 

The arc d -2 borders a domain C such that the trajectory emanating in the 
forward time direction from each switching point in C, with switching from 
control 3 to control 1, experiences an infinite number of switchings from control 1 
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to control 3 and back and then hits the semi-singular submanifold corresponding 
to the edge [v4iA 3 ] of the triangle Cl. This domain C is bounded by the arc d 2 , 
part of the arc d® 32 , and some curve c linking the point d 2 to the arc d 332 . On 
Fig. [5] the domain C is painted in green. The Poincare return map takes the 
domain C onto a subset C C C. This map leaves the arc d 2 point-wise fixed, 
and the curve c is taken to a subarc c joining the point d 2 . The image C is 
painted in dark green on Fig. [5] The curve separating C from its complement 
C\C joins the point d 2 . This curve is the image under the Poincare map $ of 
the intersection of the arc d 332 with the boundary of C. 



Figure 5: Known regions 

The inverse d* -1 of the Poincare return map takes the complement C \C, 
painted in light green on Fig. [5j consecutively to the domains painted in blue, 
red, and yellow. Recall that the Poincare map in the original space makes 
no difference between the arcs di 32 ,d 323 . Thus < f ,_1 takes the boundary of 
the domain C \ C as follows to the boundary of the blue domain. The curve 
separating the domain C from the complement C\C is taken to a sub-arc of the 
arc d 323 . The sub-arc of d 332 maps to a sub-arc of d 213 , and the intersection of 
the boundary of C\C with D 31 is mapped to the intersection of the boundary 
of the blue domain with D 3 \. 

The restriction of the inverse Poincare map ‘f’^ 1 on the blue domain, having 
the red domain as its image, or the restriction of the map $ on the red domain, 
having the blue domain as its image, is discontinuous in the topology of D 3 
Namely, the intersections of the arcs di 32 ,di 23 with the boundary of the red 
domain are glued together and are taken by $ to the dark-blue dashed line, and 
the intersections of the arcs d 213 , d 323 with the boundary of the blue domain are 
glued together and taken by to the dark-red dashed line. 

The map <h _1 takes the boundary of the red domain as follows to the bound¬ 
ary of the yellow domain. The sub-arc of d \ 32 is taken to the sub-arc of d 213 , 
and the sub-arc of d \ 23 is taken to the dashed part of the boundary of the yellow 
region. 

Note also that in an arbitrarily small neighbourhood of the point d 2 there 
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exist points which do not belong neither to the yellow nor to the green domain. 
From the results in paper m it follows that all such points, given they are 
sufficiently close to d 2 , are taken to the yellow domain by a finite power of the 
map <f>. These mappings correspond to switchings from control 3 to control 1 
and back. For each given point the number of switchings until the hit of the 
yellow region is finite, but it grows unbounded as the initial point in question 
approaches the boundary of the domain C. 

This yields a complete description of the behaviour of the Poincare return 
map in some neighbourhood of the closure of the arc <^2 - Let us establish the 
correspondence with the domains on the quotient of the switching surface with 
respect to the Fuller group g which are depicted on Fig. 4 of the paper [T?j. The 
stratum from Fig. 4 corresponds to the green domain, part of the yellow 
domain and other points in a sufficiently small neighbourhood of the point d 2 ■ 

4.4 Global structure of the Poincare map 

Let us now consider the image of the neighbourhood of the arcs d® 32 , d?23 under 
the map <F. Recall that this map glues these neighbourhoods together along 
these arcs. The image of these arcs is given by some curve cq, joining the points 
d 2 and . The dark-blue dashed line and the boundary between the dark green 
and the light green domains are parts of this curve. The curve <Tb partitions the 
disc D31 in two sub-domains. Let us denote the sub-domain containing the dark 
green region by Tj, and the sub-domain containing the light green region by R^. 

Consider the image of the neighbourhood of the arcs ^213,^123 under the 
inverse map <F -1 . This map glues these neighbourhoods together along these 
arcs and maps the arcs to some curve 07 joining the points dj” and d 2 . The dark 
red dashed line and the dashed part of the boundary of the yellow region are 
parts of this curve. The curve 07 partitions the disc P 3 1 in two sub-domains. 
Denote the sub-domain containing the green region by T/, and the sub-domain 
containing the blue region by Rf. 

Thus the map $ is continuous on the sub-domains Tf,Rf and takes them 
bijectively to the sub-domains respectively. If we extend the map $ 

continuously to the boundary of Tf, then it maps in the following way to the 
boundary of the sub-domain T},. The arc d 2 remains fixed, the arc d \ 32 maps 
to the curve cr b , and the curve <jf maps to the arc d 213 . If we extend the map 
$ continuously to the boundary of Rf, this boundary is mapped as follows to 
the boundary of the sub-domain R^. The arc d 213 is mapped to the arc di 32 , 
the arc d \ 23 is mapped to the curve cp,, and the curve 07 is mapped to the arc 
d \ 23 . The Poincare map $ is schematically depicted in Fig. [G] 
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Figure 6: Global structure of the Poincare map 

The curves <7/ and Ub intersect each other in one point. The coordinates 
of this point are solutions of a certain system of algebraic equations, because 
this point is taken to the yellow region by three-fold application of the map 
$, and the variables x,y can be written as explicit algebraic functions of the 
variables 4>, if in the yellow region. Numerical calculations of the coordinates of 
the intersection point yield \i ~ 0.174146349178, X 2 ~ 0.907326683652. 

4.5 Transitions of rotational type and transpositional type 

Consider the Poincare return map $ taking some switching point from control i 
to control j to some switching point from control j to control k. Introduce the 
following definition. 

If it happens that i = k 1 then we will call the corresponding transition 
transpositional. If on the other hand i ^ fc, and hence {i, j, k} = {1, 2, 3}, then 
we call the transition rotational. Let us denote the types of transitions by T 
(transposition) and R (rotation), respectively. 

The advantage of describing the sequence of controls on a given trajectory 
in the terms above is that it is invariant under the action of the permutation 
group 53 which shuffles the vertices of the set of admissible controls 12. 

Note that the Poincare mapping from the sub-domain Tf to Tb induces a 
transpositional transition, while the Poincare map from the sub-domain Rf to 
Rb induces a rotational transition. The indices b and / stand for backward 
and forward , respectively. A trajectory with a switching point, e.g., in the 
intersection Rf flT),, will experience a rotational transition in the forward time 
direction and a transpositional transition in the backward time direction. Note 
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also that every point in the green region C experiences during its evolution 
in forward time only transpositional transitions, until it hits a semi-singular 
trajectory in finite time, but after an infinite number of switchings. 

4.6 An attractor in the backward time direction 

In P~7j the existence on D^i of a repelling fixed point Fr of the Poincare map 
was proven. This fixed point corresponds to the three-link periodic cycles in the 
quotient space with respect to the Fuller group g am Theorem 3). Further, an 
hyperbolic periodic orbit of the map $ was found which consists of two points 
Fr’Fr an d corresponds to the six-link periodic cycles from [17] . Theorem 5. 
On Fig. [7] the point Fr and the pair of points Fr , Fft are marked by blue and 
red crosses, respectively. 



Figure 7: Regions I and lb 


The self-similar trajectories corresponding to the periodic orbits enumerated 
above experience a cyclic sequence of all three controls. Hence the corresponding 
transitions are of rotational type, and all three points Fr, Fr, Fr lie in the 
intersection of the sub-domains Rf,Rb- 

The point Fr is repelling in the forward time direction and hence attract¬ 
ing in the backward time direction. Hence there exists a neighbourhood of the 
point Fr such that every point of this neighbourhood tends to Fr under re¬ 
peated applications of the inverse Poincare map <f> _1 and experience exclusively 
rotational transitions on the way. Denote the largest such neighbourhood by I. 
Clearly the region I is a subset of the domain 74. 

Numerical computations revealed that one of the branches of the stable 
submanifold of the hyperbolic periodic orbit (Fr, Fr) lies in region I. It follows 
that both points Fr, Fr lie on the boundary of region I, and region I is actually 
bounded by the unstable submanifolds of the periodic orbit (Fr, Fr). Hence 
region I is nothing but the intersection of the strip lying between the unstable 
manifolds of the periodic orbit (Fr, Fr) and the sub-domain R The remaining 
parts of the boundary of region I consists of parts of the arc d ® 23 and the curve 
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The intersections of region I with the blue, yellow, red, and green domain is 
painted in magenta on Fig. [ 7 ) Denote the image of region I under the inverse 
Poincare map by Ic. Since the fixed point Fp> is attracting in backward time, 
the region Ic is a subset of region I. The intersection of region Ic with the blue, 
yellow, and red domains are painted in dark magenta on Fig. [ 7 ] Denote the 
complement of region Ic in region I by la. The intersection of region la with 
the red and green domain is painted in light magenta on Fig. 0 The boundary 
between the regions la and Ic is part of the curve 07. When region Ic is mapped 
by <f> on its image, this boundary arc is taken to a part of the boundary of region 
I, namely a part of the arc d 32 3. This latter sub-arc of d\ 23 is also part of the 
boundary of region Ic. The map $ takes it on the boundary of region I, namely 
on a part of the curve a b . 

It follows from the preceding that the image of region I under the map <f> is 
the union of region I with the image of region la. Let us denote this image d>[/o] 
by lb. The intersections of region lb with the blue, part of the yellow, and green 
domains is painted in cyan on Fig. [T] When region la is mapped to its image by 
the Poincare map <f>, then the sub-arc of the curve 07 on the boundary of region 
la is taken to a sub-arc of d 213 , an d th e sub-arc of a b is taken in the interior of 
the dark green region. Note that the intersection of the curve 07 with region 
lb lies entirely on the boundary of the yellow region, but the boundary of the 
yellow region does not contain the intersection point of the curves 07, a b . 

4.7 Intermediate regions 

In the preceding subsection we established that the points lying in the blue and 
red domain are taken to the green domain by one or two applications of the 
Poincare map <f>, respectively, from where they join the semi-singular trajectory 
after an infinite number of transitions of transpositional type. On the other 
hand, the inverse map takes the points of region lb to region I, where they 
further to the point Fr, experiencing transitions exclusively of rotational type. 
In this subsection we consider the remaining points of the disc D 3 i. 

Every point of D 31 which does not lie in the closure of the green, blue, or red 
domain or one of the regions I, lb, must lie in the closure of one of the following 
four regions. 

Region II is bounded by a sub-arc of d 332l part of the boundary of the red 
domain, part of the boundary of region Ic, and part of the boundary of the blue 
domain. Region III is bounded by part of the boundary of region I, part of the 
boundary of the green domain, part of the boundary of region lb, and part of 
the boundary of the blue domain. Region IV is bounded by a sub-arc of d 323 , 
part of the boundary of the green domain, part of the boundary of region la, 
and part of the boundary of the red domain. Region V is bounded by part of the 
boundary of region lb, part of the boundary of the green domain, and a sub-arc 
of d 213 . The location of these regions is schematically depicted in Fig. [H see 
also Fig. [ 9 ] 
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Figure 8: Schematic location of the regions 



Figure 9 : Regions II,III,IV,V 

Let us now consider the dynamics on these intermediate regions which is 
induced by the Poincare map. Note that both curves 07 and ct;, partition region 
III in two parts, respectively. Denote the subsets delimited by the curve 07 by 
III1 and Illr, respectively, and the subsets delimited by the curve 07, by IIIu 
and Hid, respectively. Here region III 1 borders the blue domain, Illr the green 
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domain, IIIu borders region lb, and Hid borders region I. Note also that the 
curve a/ divides region V in two parts, which we denote by VI and Vr. Here 
region VI borders the arc c? 213 j an d Vr the green domain. 

Then the Poincare map $ maps region II bijectively onto region Hid, here 
the part of the boundary of the blue domain is taken to a part of the boundary 
of the green domain, the part of the boundary of the red domain is taken to the 
part of the boundary of the blue domain, the part of the boundary of region Ic 
is taken to the part of the boundary of region I, and the sub-arc of g? 5 32 to a 
part of the curve ov The corresponding transition is of rotational type. 

The Poincare map maps also region IV bijectively onto region IIIu, here 
the part of the boundary of the light green domain is taken to the part of the 
boundary of the dark green domain, the part of the boundary of the red domain 
is taken to the part of the boundary of the blue domain, the part of the boundary 
of region la is taken to the part of the boundary of region lb, and the sub-arc of 
di 23 to a part of the curve cp,. The corresponding transition is of transpositional 
type. 

From Fig. [G] it is clear that the image of region VI is bounded by the arcs 
di 32 and di 23 - Here the intersection of the boundaries of regions lb and VI is 
taken to some curve joining these two arcs. This curve cannot intersect region I, 
since the image of region I under the inverse map d*” 1 does not intersect region 
VI. The endpoints of this curve he in the blue and yellow domains, respectively, 
hence the endpoints of the image of this curve lie in the green and red domain, 
respectively. Therefore $ takes region VI to the union of some subsets of the 
red and green domains and a subset of region IV. The corresponding transition 
is of rotational type. 

Now we shall consider how $ acts on region Vr. Consider the image of re¬ 
gion V under the inverse map 3> -1 . From the preceding it follows that $~ 1 [V] is 
bounded by the boundary of the green domain, the curve 07, and the boundary 
of region la. In particular, this image contains the region Vr. Hence $ takes 
region Vr to a subset of region V. The corresponding transition is of transposi¬ 
tional type. 

From the preceding paragraph it follows that region Mr is contained in the 
preimage <I> _1 [V] of region V. Hence region Mr is taken by $ to a subset of 
region V. The corresponding transition is of transpositional type. 

Consider now the image of region Ml under $. The intersection of the 
boundaries of region Ml and the blue domain is taken to part of the boundary 
of the green domain. The intersection of the boundaries of regions Ml and Ic is 
taken to part of the boundary of region I. The part of the curve <7/ bordering 
region Ml is taken to a sub-arc of d 323 . The endpoints of this part of the curve 
<7/ lie in the blue and yellow domain, respectively. Hence the endpoints of the 
sub-arc of d 323 lie in the blue and red domain, respectively. Therefore the image 
$[////] is a union of region II and subsets of the red and blue domains and region 
IV. The corresponding transition is of rotational type. 

The images of regions II, Ml, Mr, IV, VI, and Vr under the map $ are 
schematically depicted in Fig. [10] On Fig. [TU] on the right it can be seen 
that region IV is divided by the images of regions III and V into three strips 
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IVa,IVb,IVc. Here we define region IVa as the intersection of region IV with the 
image of region III, region IVc as the intersection of region IV with the image 
of region V, and region IVb as the complement of IVa and IVc in region IV. 



Figure 10 : Images under $ of regions II,III,IV,V 


Let us now consider the preimages of regions II-V under the map $. The 
inverse map $ _1 takes region II to a subset of region III 1 . Here the sub-arc of 
^123 bordering region II is taken to an arc on the curve tjf. The parts of the 
boundaries of the blue and red domains are taken to parts of the boundaries 
of the red and yellow domains, respectively, and the part of the boundary of 
region Ic to another part of the boundary of region Ic on the other side. The 
corresponding transition is of transpositional type. 

Consider the preimage of region IV under $. The sub-arc of d ? 32 bordering 
region II is mapped by $ -1 to a sub-arc of ($13, the parts of the boundaries 
of the green and red domains are taken to parts of the boundaries of the blue 
and yellow domains, respectively, and the part of the boundary of region la is 
mapped to a part of the boundary of region Ic. Hence the preimage of region IV 
is the union of subsets of regions III 1 , VI, and lb. Denote this subset of region 
VI by Va. The corresponding transition is of rotational type. 

The preimages of regions IIIu and Hid are the regions IV and II, respec¬ 
tively. The corresponding transitions are of transpositional and rotational types, 
respectively. 

Finally, the preimage of region V has been described above. It is the union of 
the regions Illr and Vr and a subset of region lb. The corresponding transition 
is of transpositional type. 

The preimages of regions II, Hid, IIIu, IV, and V under the map $ are 
depicted schematically in Fig. (TT) 
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Figure 11: Images under $ 1 of the regions II,III,IV,V 


The dynamics of the Poincare map on the regions II-V can be described by 
the oriented graph presented in Fig. Q2J The outgoing edges denote transitions 
into points of the red, blue, or green domains, where they are attracted by the 
semi-singular trajectory. The incoming edges denote transitions from points in 
the regions I or lb. Under iterations of the inverse map $ _1 these points are 
attracted by the fixed point Fr. The letters t,r denote the type of transition, 
transpositional or rotational, respectively. 



t 


Figure 12: Transition graph 


4.8 Resolution of the dynamics on region V 

Consider the intersection of region V with the images of regions III and V on 
the one hand, and with the preimages of regions IV and V, on the other hand. 
These intersections are schematically depicted on the right-hand side of Fig.fl3l 
On the left-hand side of Fig. Q3] one sees the exact locations of the points which 
are denoted by dots on the right-hand side. 
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Figure 13: Dynamics on region V 

From Fig. [13] it becomes clear that region Va is a subset of the intersection of 
the image of region III with region V. Hence every point which comes into region 
V from outside of region III can never return to any of the regions II, III, and 
IV. Such a point either lies in the intersection of region V with the blue or the 
yellow domain, or hits the yellow domain after a finite number of applications 
of the map <F, where the corresponding transitions are of transpositional type 
only. From the yellow domain it gets after three applications of $ to the green 
domain, from where it is attracted by the semi-singular trajectory. 

The graph on Fig. [12] implies that every trajectory that is not eventually 
attracted in forward time direction by the semi-singular trajectory, and in back¬ 
ward time direction by the fixed point Fr, has to pass an infinite number of 
times through the closure of region III. Hence when studying these trajectories, 
we can restrict our attention to the mapping of the closure of region III on itself. 

The graph on Fig. [12] implies that this mapping can proceed in three dif¬ 
ferent ways. Either through two transitions of rotational type passing region 
II, or through two transitions, of transpositional and rotational type, respec¬ 
tively, passing region IV, or finally through three transitions, of transpositional, 
rotational, and again transpositional type, respectively, passing regions V and 
IV. Let us denote these possibilities of the mapping of the closure of region 
III to itself by A, B , C 1 respectively. Hence every trajectory which is not even¬ 
tually attracted in forward time direction by the semi-singular trajectory, and 
in backward time direction by the fixed point Fr, can be represented by a bi¬ 
laterally infinite sequence of symbols A, B 1 C. In the sequel we shall argue in 
terms of transitions of type A, B 1 C instead of transitions of transpositional and 
rotational types. Here the former are actually composed by the latter. 

Consider a point in the closure of region III. When applying recursively the 
map $ to this point, it will pass a number of times rif through the closure of 
region III. Here n/ may be finite or infinite. The corresponding transitions will 
be labeled by symbols i\, 1 %,..., i n , € {A, B,C}. When applying the inverse 
map <1> _1 to this point, it will pass a number of times rib through the closure of 
region III, where nb also may be finite or infinite. The corresponding transitions 
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will be labeled by symbols ji, J2, • ■ •, j nt , G {A,B,C}. We then associate to the 
initial point the two-sided sequence jn b jn b -i ■ ■ ■ ■ ■ • i nf - Here the period 

separates transitions which lie in the future (with respect to the time flow defined 
by the iterations of $) from transitions which lie in the past. 

On the other hand, let us associate to the two-sided sequence 


jn b jn b —l ■ ■ ■ jl-ili? ■ • ■ inf 

the subset of points in the closure of region III which experience the transitions 
iii * 2 ; • • • i E {A,B,C} under iterations of the map 4>, and the transitions 
ji,j2, ■ ■ ■ i jn b under the iterations of the inverse map 4> _1 . Thus, e.g., the set 
A.B is the intersection of the closure of region III with the preimage of transition 
B and the image of transition A. 

The set A. then coincides with the closure of region Hid, the set B. is the 
image of the region IVc, and the set C. is the image of region IVa. On the other 
hand, the set .A is the closure of the preimage of region II, the set .B the closure 
of the intersection of the preimage of region IV with region III, and the set .C is 
the preimage of the region Va. The sets .A, .B, .C are depicted on the left-hand 
side of Fig. [IH the sets A., B., C. on its right-hand side. The transitions A, B , C 
map the former to the latter, respectively. 



Figure 14: Dynamics on region III 

Note that the sets .A, .C intersect each other in some arc of the curve 07 , 
which is at the same time part of their boundaries. Both sets border region I 
and the red domain. The set .A borders also the yellow domain, and the set 
.C borders the preimage of the intersection of the yellow domain with region V. 
On the other hand, the set .B borders the regions I and lb and the blue and 
yellow domains. The set B. borders region lb, the blue and green domains, and 
the image of region IVb. The set C. borders the set A. along the curve cp,, and 
both sets border the blue and green domains. The set C. borders also the image 
of region IVb, while A. border region I. 

Note that all three sets on the left-hand side of Fig. [IT] intersect all three 
sets on the right-hand side. Hence any type of transition can follow any type. 
Moreover, each of the transitions A,B,C features a fixed point. On Fig. [14] 
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these fixed points are represented by a red, blue, and green cross, respectively. 
The fixed points of A and B correspond to the six-link and the four-link cycle, 
respectively, which have been found in the paper and the fixed point of C 
corresponds to a nine-link cycle (which has been discovered in 2002). 

4.9 Resolution of the dynamics of transitions of type B 

From Fig. [15] it is clear that the sets .AA, .AC, .CA, .CC have an empty 
intersection with the set B.. Hence the sequence of symbols corresponding to a 
trajectory of the system cannot contain any subsequence of the form Bij , where 
i,j £ {A,C}. By induction it follows that if the sequence somewhere contains 
two consecutive symbols of the set {A, C}, then the symbol B can occur only 
to the right of this pair. 



Figure 15: Dynamics with participation of transition B 


The sets .BA, .BC are subsets of B ., and the sets .BBA, .BBC intersect all 
three sets A., B., C.. The sets . BBBA, .BBBC are also subsets of B ., and the 
sets .BBBBA, .BBBBC are subsets of A. (see Fig. fl6l) . The sets .ABBBBA, 
.ABBBBC are also subsets of A., whereas .ABBA, .ABBC, .CBBA, .CBBC 
have a non-empty intersection with the set B. (see Fig. ED). 
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Figure 16: Dynamics with participation of transition B 


The sets .BABBA,.BABBC ,. BCBBA,.BCBBC , which are painted in 
blue on Fig. [TT1 are subsets of the union .BA U .BC and hence also subsets of 
B.. However, the sets .BBABBA, .BBABBC, .BBCBBA, .BBCBBC, which 
are painted in red on Fig. [TT1 are already only subsets of A., and the sets 
.ABB ABB A,.ABBABBC ,. ABBCBBA,.ABBCBBC , which are painted in 
green on Fig.[l71 do not intersect B.. 



Figure 17: Dynamics with participation of transition B 

From the preceding it follows that if the sequence of symbols contains a 
subsequence of the form Bi, where i £ {A, C 1 }, then this subsequence can be 
continued to the left only by one of ... AABBBBi, ... ABBjBBi, ... BBi , 
where j £ {A, C} and the dots denote a subsequence of arbitrary length, but 
consisting only of symbols in {A , C}. 

In particular, a sequence which is infinite on both sides either (i) consists 
entirely of symbols B , in which case it corresponds to the fixed point of the 
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transition B, i.e., the four-link periodic cycle, or (ii) it contains a subsequence 
consisting entirely of symbols B and stretching out infinitely far to the right, 
a subsequence not containing any symbol B and stretching out infinitely far to 
the left, and between these two infinite subsequences maximally six additional 
symbols, or (iii) it does not contain the symbol B at all. Therefore, if the 
corresponding trajectory of the dynamics on region III is not the fixed point of 
transition B, then it hits in backward time direction the union of the sets A.A, 
A.C, C.A, C.C and remains there indefinitely. 

4.10 Dynamics of the transitions of types A and C 

We shall now consider the dynamics of the transitions A, C on the union of the 
sets A.A, A.C, C.A, C.C, or more precisely, on the closed region Illb defined 
as the union of the sets A.A, A.C, C.A, C.C and the closure of the intersection 
of the red domain and region III. Hence region Illb borders region I, the yellow 
domain, the image of region IVb, and the preimage of the intersection of the 
yellow domain with region V. The region Illb is depicted on the left-hand side 
of Fig. [Island on the right-hand side of Fig. fl9l 



Figure 18: Image of region Illb under F 




Figure 19: Image of region Illb under F 1 
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Let us define mappings F, F _1 : Illb —> D 3 i as follows. Let x be a point in 
the strip between the yellow domain and the curve cr/. We then define its image 
F(x) as the image of x under the transition A, i.e., under the power <I > 2 of the 
Poincare map. Both applications of $ induce transitions of rotational type. Let 
now x be a point in the strip between the preimage of the yellow domain and the 
curve <jf. We then define its image F(x) as the image of x under the transition 
C, i.e., under the power <f > 3 of the Poincare map. The three applications of $ 
induce transitions of transpositional, rotational, and again transpositional type, 
respectively. The map F can be extended continuously on the intersection of 
region Illb with the curve ct/. Let x be a point in the strip between region I 
and the curve cp,. We define its image -F - 1 (x) under the inverse map F~ x by 
the preimage of x under the transition A , i.e., as $ _ 2 (x). Here both transitions 
generated by the application of <& -1 have rotational type. Let finally x be a 
point in the strip between the image of region IVb and the curve cp,. We define 
F-\ X ) as the preimage of x under the transition C, i.e., as < F _ 3 (x)- Here 
the three transitions generated by the application of <f > _1 have transpositional, 
rotational, and again transpositional type, respectively. The map F~ x can be 
extended continuously on the intersection of region Illb with the curve op. 

The map F takes an arc on the curve ct/ to an arc on the curve op, and 
T 1-1 vice versa, an arc on op to an arc on o’/. The images of these arcs do not 
intersect region Illb, and the intersection of region Illb with F[IIIb\ as well as 
the intersection of region Illb with F _ 1 [/// 6 ] both consist of two connection 
components, respectively. Thus the topology of the map of region Illb on itself 
is equivalent to the topology of the map in the classical Smale Horseshoe. The 
images of region Illb under the maps F, F~ x are depicted in Fig. [TH1 Fig. [El 

On Fig. lT8llT9l the fixed points of F are represented by red and green crosses. 
These two points are the fixed points of the transitions A and C, respectively, 
and have been already depicted in Fig. [L4] The blue crosses in Fig. [El [El 
denote a periodic orbit of the map F consisting of two points, namely the fixed 
points of the compositions AoC and Cod. This orbit corresponds to a 15-link 
periodic cycle on the quotient space M + /g , which is invariant with respect to a 
permutation of order 3 in S 3 . All three periodic orbits are of hyperbolic type. 
The stable eigenvalues are approximately equal to 


0.789389405489713, -0.195427788394708, -0.064093706436160, 
respectively, and the unstable eigenvalues to 

25.416902415438734, -2.553017991174260, -73.29685207454, 

respectively, for the fixed points of the mappings A,C,A o C. 

Let us remark that the map F is continuously differentiable and even alge¬ 
braic on a dense subset of region Illb. This subset is the set of points which 
under repeated application of the Poincare map $ eventually hit the interior 
of the green domain, without leaving the interior of the disc D 31 on their way. 
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Therefore any such point can be obtained by repeated application of the inverse 
map to some point in the green, or equivalently, blue, domain. 


5 Chaotic dynamics of the Poincare map 

In this section we prove the main theorem of the first part of this paper, on the 
semi-conjugateness of the dynamics of the optimal synthesis in model problem 
ra with equilateral triangle with a certain Markov chain. We find bounds on 
the Hausdorff and Minkowski dimensions of the non-wandering set and compute 
the topological entropy. 

5.1 Bilipschitz property of the Poincare return map 

The main tool used in the investigation of the optimal synthesis of problem © 
in the previous sections was Theorem [l] on the passage to the adjoint variables, 
or more precisely, the map E allowing this passage. Therefore a proof of the 
bilipschitz property of the map $ needs to commence with an investigation of 
this property for the map E. 

In Theorem [I] we have shown that in a broad class of optimal control prob¬ 
lems, including model problem (©, the map E is locally Lipschitz and bijective. 
In general, the inverse map E _1 does not need to be even locally Lipschitz for 
the general problem ra¬ 
in the next lemma we show that the map E in model problem ra is locally 
bilipschitz in the neighbourhood of any point not lying on the submanifolds Ay . 

Lemma 17. Consider a point q 0 = (cco,2/o) which does not lie on any of the 
planes Ay, i.e., x$ (f U Ay or yo U Ay. Then there exists a neighbourhood of 
qo on which the map E satisfies a bilipschitz property. 

Proof. The proof consists in obtaining a lower bound on the difference B(qi) — 
B(qo) — B'(qo)Aq and an application of Lemma [7] Here we use the usual nota¬ 
tions qi = (: Vi, yi) and A q = (Ax, Ay) = q 1 - q 0 . 

Let Xq (t) and Xi (t) be two arbitrary trajectories emanating from points 
Xo and X\ , respectively. For an arbitrary number A £ [0,1] we define X x (t) = 
(1 — A)Ao(t) + XXi ( t ). Then the functional J defined in © satisfies the relation 
(see Lemma [2J 

(1 - A) J(X 0 ) + XJ(Xi) - J(X x ) = A(1 - X)J(AX), 

where A X(t) = Xi(t) — Xo(t). If the trajectories Xq and X\ are optimal and 
-Xj(O) = Vi, i = 1, 2, then 

(1 - A) B(qo) + A B{ qi ) - B(q x ) > A(1 - X)J(AX), 

where q x = (1 — A)qo + Xqi. Since both sides of the inequality coincide at A = 0, 
we have after differentiation with respect to A at A = 0 that 
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B(i 7 i) - B(q 0 ) - B'(q 0 )Aq > J(AX). 

It rests to estimate J(AX) from below. The simplest way to do this, namely 
J(AX) > B{Aq) yields an insufficient result of an order which is too high, 
namely B(Aq) > A Ax 2 + BAy 5 . In general, one cannot have a better bound 
on the Bellman function at an arbitrary point. 

However, if the point qo does not he on Ay, we can have a better bound on 
the functional J(AX). Assume first that go ^ U(Ay x Ay) is not a switching 
point. This means that in a sufficiently small neighbourhood V of the point 
go there exists r > 0 such that for time instants t < r all optimal trajectories 
emanating from points q± £ V are steered by the same control uq residing in 
one of the vertices of the triangle SI. Indeed, the lift of the optimal trajectory 
(x(t), y(t), ft{t)) emanating from (go,£’(go)) intersects the switching sur¬ 
face S in some point ( 2 /, y', ft , ft) (go, -E’(go)) at some time instant to > 0 for 
the first time. Since the surface S is closed, we have that for every e > 0 there 
exists 5 > 0 such that the trajectory of Hamiltonian system m emanates from 
a ^-neighbourhood of the point (go, E(q 0 )), then it intersects S for the first time 
at a time instant not smaller than t 0 ~ £• Now note that by virtue of Theorem[T| 
the map E is continuous, hence if gi is close to go, then also (qi,E(qi)) is close 
to (g 0 ,-E(g 0 )). 

For all t £ [0, r] we have the equation AX(t) = Ax+tAy, because the control 
on the optimal trajectories emanating from the points go and gi coincides on 
this time interval, given the point gi lies in a neighbourhood of go- Hence there 
exists C' > 0 such that 


B(qi)-B(q 0 )~B l (q 0 )Aq> J{AX) > \Ax+tAy )\ 2 dt > C(\Ax \ 2 + \Ay\ 2 ). 

Application of Lemma 0 then yields the proof of the assertion of Lemma [T71 
for all points go not lying on a switching surface. Suppose, on the contrary, that 
the point go lies on a switching surface. In order to get a lower bound on J( AX) 
we have to move on all optimal trajectories emanating from a neighbourhood 
of go by a small time interval to > 0 and to repeat the reasoning above. □ 

HencJ^l. 

1. The map E : M —> N is a locally Lipschitz homeomorphism, which is 
bilipschitz at all points outside of U(Ay x Ay); 

2. The map q i-p (g, E(q)) is a locally bilipschitz homeomorphism from M to 
M +; 

3. The map (q,E(q)) 1 —> E(q) is a locally Lipschitz homeomorphism from 
M + to N, which is bilipschitz at all points outside of U(Ay x Ay). 

11 On the sets M, N, M+ the structure of a metric space is defined in a natural way by the 
ambient vector space Jvt = T* M = M X N. However, the set M+ is not a smooth submanifold 
of Ad. 
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Figure 20: Graph T 


All these maps retain the enumerated properties after factorization with 
respect to the action g. 

Let us now consider the bilipschitz property of the Poincare map <f>. Let 
x G (S fl M + )/g be a point which lies neither on |J A*, /g nor on £ 123 /( 7 . By 
virtue of Lemma [5] the image $(:r) G (Sfl M+)/g is then well-defined. Let 
us assume that $(x) ^ £ 123/(7 and $(a:) ^ IJAf^/g. Then the projection of 
the trajectory of Hamiltonian system (0 through x intersects the switching 
surface transversally both in the point x and in the point <f>(:r). The map 
T : S/g — > S/g is hence well-defined on a neighbourhood of a; in S/g and is 
a local homeomorphism of a neighbourhood of x in S/g onto its image. We 
then get immediately that the restriction of $ on (£ fl M + ) /g is bilipschitz in 
a neighbourhood of x. By virtue of the bilipschitz property of the map E we 
obtain the following lemma. 

Lemma 18. The Poincare return map $ : (£ fl M+)/g — > (£ fl M + )/g is 
bilipschitz in the neighbourhood of every point x such that neither x nor $(a;) 
do not lie on £ 123/(7 or (J Af^/g. The bilipschitz property is conserved if we 
consider the map $ as acting in the spaces M/g or N/g by identification by 
virtue of the map E. 

Corollary 10. The map F : A.AU A.CVJC.APtC.C —> A.AU A.CVJC.APtC.C, 
which was defined in Subsection \4-10\ is bilipschitz. 

5.2 Conditions of Lipschitz hyperbolicity 

Let us study the dynamics of the map F in detail. As was shown in the previous 
section, we may choose disjoint rectangular neighbourhoods B 1 , H 2 , £ 3 , B 4 of 
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F 

Ux. 


Uy 

Vy 

Vy 


F n 

21 

25.6 

0.57 

0.739 

0.792 

0.15 

F 31 

20.3 

22.4 

0.37 

0.4 

0.48 

0.14 

F \2 

22.5 

34 

3.1 

0.72 

0.785 

0.5 

F 32 

21 

25 

1.9 

0.4 

0.48 

0.07 

F'23 

3.08 

3.28 

0.06 

0.07 

0.18 

0.06 

F 43 

3.11 

3.28 

0.09 

0.176 

0.019 

0.0591 

F 24 

2.35 

2.9 

0.75 

0.1 

0.22 

0.1 

F 44 

2.3 

2.9 

0.8 

0.175 

0.21 

0.1 


F~ l 

Uv 

Ih 

If 

%lL 


Xy 

F n 

1.26 

1.36 

0.098 

0.039 

0.049 

0.37 

F31 

2.09 

2.51 

0.16 

0.0444 

0.049 

0.045 

F _1 

r 12 

1.342 

1.384 

0.0195 

0.029 

0.045 

0.18 

F32 

2.1 

2.5 

0.0061 

0.04 

0.048 

0.2 

F23 

5.6 

14.1 

0.25 

0.305 

0.323 

0.171 

F _1 

r 43 

5.3 

5.66 

0.0094 

0.303 

0.323 

0.151 

F24 

5 

14.4 

0.4 

0.37 

0.394 

1.41 

F44 

4.6 

5.6 

0.21 

0.364 

0.381 

1.55 


Table 1: Constants u x , u x etc. for the maps F and F 1 



Bi 

b 2 

B 3 

b 4 

c 

0.00743 

0.0225 

0.022 

0.0488 

d 

0.146 

0.321 

0.103 

0.374 


Table 2: Constants Ci and di for the maps F and F 1 on the rectangles Bi. 


the sets A.A, A.C, C.A, C.C and corresponding coordinates (Xi,Yi) on Bi, 
i = 1,2,3,4. Denote by Fij the restriction of F on the subset of those points 
in Bi which are mapped to B } by one iteration of the mapping F, i.e., F^ = 
The maps F t] are defined in accordance with the graph T on 
Fig. (50) the map F^ is defined if and only if there exists an edge linking the 
corresponding vertices in the graph T. 

Denote by Uij(Xi, Yi) and Vij(Xi,Yj) the first and the second coordinate of 
the image Fij(Xi,Yi) in the rectangle Bj, respectively, such that Fij(Xi,Yi) = 
(Uij(Xi, Yi), Vij(Xi, Yj)). According to the previous subsection the map F\j 
satisfies the inequalities 


| uJi\AX\ < 

r v j '|Ay| < 


Uij (X t + AX, Yi ) - Ui) (Xi, Yi) 

b 0 (.V,.V, • AT) r„!.Y,.V,) 

| Vij(Xi, Yi + AY) - Vij{Xi, Yi)) 
| VijiXi + AX, Yi) - Vij{Xi,Yi)\ 


< u? j \AX\, 

< u? j \XY\, 

< vj j \&n 

< v? j \AY\, 
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F 

o-i 

02 

03 

04 

05 

06 



A 


0.797 

0.249 

0.0748 

0.0238 

0.0201 

0.0883 



a; 

7 

0.734 

0.137 

0.0178 

0.00462 

0.0304 

0.0271 



a 

7 

0.739 

0.701 

0.643 

0.695 

0.674 

0.673 




Table 3: 

Constants A CT , A^ 

, and a~ 

for the map F. 


F~ 

-i 


0i 

02 

03 

04 

05 

06 

K 

0.0484 

0.46 

0.000785 

0.00829 

0.000403 0.0162 

K + 

0.0386 

0.288 

0.000341 

0.00294 

0.0000864 0.0117 

a 

f 

j 

0.931 

0.625 

0.895 

0.822 

0.835 

0.927 


Table 4: Constants A+, , and a + for the map F 1 . 

at every point ( X i: Yi ) e and for every feasible variation (AA, AY), 

where u., u x IJ etc. denote positive constants whose numerical values are given 
on the left-hand side of Table [Jffi Moreover, the inverse mappings F~- 1 sat¬ 
isfy the same sort of inequalities with other constants x^ lJ , x u u etc. whose 
numerical values are given on the right-hand side of Table [lj 

Note that the values of the constants u^, Ux_ etc. depend on the choice of 
coordinates Xi and Y t . The values given in Table |T] are valid for coordinates X, 
and Yi explicitly defined by 


Ad = A 3 


= 3636.746266824261x1 - 4481.163862258486x2 + 

+ 3855.150222649458xi - 9508.946041171987xiX2 + 

+ 5858.268250499643x1 + 1403.048095297728x? - 

- 5062.144152379995xiX2 + 6220.813348898283xixi - 

- 2553.628003178778x1, 


X 2 = X 4 


15(—434.858450049792xi - 1178.182552029670x2 - 
120.018213318585xi + 1034.238125839267xiX2 + 
1197.949813776455x1 + 294.581582485282x? - 
56.750230435401x?X2 - 570.219924579354xiX2 - 
401.871126962896x1), 


12 The numerical values in the tables are rounded in the right direction, i.e., such that the 
inequalities are not violated. 
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Y x =Y 2 


= 3.239086633898x1 - 908.305710474105x2 - 

- 19.706205900010xi - 2.212072200027 xiX2 + 

+ 1018.022649246405x2 + 6.183323553959x? + 

+ 19.428876703623xiX2 - 1.286781359343xiX2~ 

- 379.719994048579x1, 


Y 3 =Y 4 = 


+ 

+ 


7714.46037676461xi + 57227.20693518666x2 - 
1394.23893421762xi - 16419.64632698288xiX2 - 
61359.76468135373x2 + 68.70577470503x? + 
1493.59693767978xiX2 + 8734.94321423633xiX2 + 
21935.18377642208x1- 


Since the rectangles Bi define a pre-Markov partition of the seiF^I B = UiBi 
with respect to the maps Fij, we can apply the apparatus developed in jlO] for 
the investigation of Lipschitz hyperbolic dynamical systems. The main condition 
for the applicability of this technique for the map F is the condition of Lipschitz 
hyperbolicity: 


Definition 7. We say that a dynamical system on a pre-Markov partition B = 
UiBi satisfies the conditions of Lipschitz hyperbolicity if the following conditions 


hold: 


(I) There exist constants c, > 0, i = 1,2,3,4, such that the inequalities 


ILx Ci XL' 


y 


Ci Vy ■ 


u x J - Ci U qi 


< Cj for each directed edge (ij) in f 


hold; 

(II) The constants C; > 0 can be chosen such that for every elementary cvclcFI 
a = (* 1*2 ... iki l), V £ {1, 2,3,4} the condition 


i'2 ^12^3 ' ‘ ' 


< 1 , 


(17) 


holds, where 


A b = 


U X J Vy 


+ Uy 


Ux 1 U-iUy 


(18) 


13 For the exact definition of pre-Markov partition, see m ■ 

14 A cycle (io, • • in a directed graph T will be called elementary if it contains each 

vertex of the graph at most once. 
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The map F satisfies the conditions of Lipschitz hyperbolicity on B = U iBi 
with the constants Cj given in Tabic [2] Indeed, condition (I) can be verified by 
direct substitution, and condition (II) can be verified by substitution into all 
elementary cycles. There are six elementary cycles in the graph T, namely 


<7i = (A.A; A.A), (t 2 = (C.C; C.C), cr 3 = (A.A; A.C; C.A; A.A), 
<t 4 = (C.C; C.A; A.C; C.C), a 5 = (A.A; A.C; C.C; C.A; A.A), 


and < 7 g = (A.C; C.A; A.C)). The numerical values of the products (fI71) for the 
map F are given in Tabic [3] 

In order to obtain bounds on the dimensions we shall need also the constants 
a~ and A“ = X~ i2 'X~ l3 ' ■ ■ ■ A";/, where 


^ = 


U 7 , 13 
c Vy 


U U X 


<K 0 


and 


L-l tly 


log K 

log Aa 


(19) 


The numerical values of the constants a~ and X'~ are also given in Table [3] 
The Lipschitz hyperbolicity of F implies that B = U iBi is a pre-Markov 
partition also for the inverse map F~ x , see {TO]. Note that in the general case 
it is not true that the Lipschitz hyperbolicity of F implies that the inverse map 
-F -1 satisfies conditions (I) and (II) with some constants di > 0. However, in 
our case the constants di for the map F~ 1 indeed exist, their numerical values 
are given in Table [2] 

Thus we have proven the following lemma. 


Lemma 19. The maps F and F~ 1 act on a pre-Markov partition B = U iBi 
and satisfy the condition of Lipschitz hyperbolicity on it with positive constants 
Ci and di, i = 1,2, 3,4, respectively, whose numerical values are given in Tafc/J^l 

n 

Note that for the map F _1 the constants Xf' = A^^A^ ... A^^ 7 , A+ = 
Afp 2 A ^?: 3 ■ • ■ 'V 4 .i 1 ) anc l a t , which are given in Table U have been computed by 
the formulas 


n, o ' 1 _ 77—7* 7 j —ji- 

)/ yv yu -^v 


< x ti 


Vv :I X u 31 + y u Jl x v Jl log A+ 

=---—- ry ' — --— 

1 z 7 1 4 7 a I / * 

Vv djX v log X(j 

( 20 ) 


15 The values of the constants and d{ have been found on a computer with the help of a 
not very sophisticated optimization procedure. We shall not detail it here, because the reader 
may verify the validity of the constants Ci and di by direct substitution. 
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5.3 Conjugation with the topological Markov chain 

We want to prove that the Lipschitz hyperbolic dynamic system F : B —> B is 
semi-conjugated with the two-sided topological Markov chain Ep. For this we 
use the following theorem from uni. Here S(F) denotes the set of points x € B 
whose images F n [x ) are in B for all n £ Z. 

Theorem 3 ( jlOj . Theorem 1). Suppose that the Lipschitz dynamical system 
F : B —> B possesses a pre-Markov partition B = U ^ =l Bi with an oriented graph 
r. Suppose also that the map F is invertibl^ 1. and that 


u x l3 Vy lJ — u y lJ v x i:> > 0 for each directed edge ( ij ) of the graph T, 

and that F and F~ x satisfy the conditions of Lipschitz huverbolicit ¥^ I with con¬ 
stants Ci and di, respectively, and that Cidi < 1 for 1 < i < N. Then there 
exists a homeomorphism Ty : Ey —> S(F) which semi-conjugates F with the left 
Markov shift l, \Fy o l = F o \I/p. Moreover, the non-wandering set NW(F) is a 
subset of S(F), and if the graph T is strongly connected, then NW (F) and S(F) 
coincide. 

As a direct consequence of this theorem we obtain the following lemma on the 
structure of the non-wandering set of points of the map F under consideration. 

Lemma 20. The Lipschitz dynamical system F : B -A B is semi-conjugated 
with the left shift l on the topological Markov chain Ep of bilaterally infinite 
paths on the graph T (see Fig. \2Cfy . i.e., there exists a continuous embedding 
’Fp : Ep —y B such that F o vFp = 'Fp o l. Moreover, S(F) = NW(F) = 'Fp(Ep). 

Proof. It is trivial to check the conditions of the theorem for the map F under 
consideration, since the constants c* and di have already been calculated (see 
Lemma fTTH) . Strong connectedness of the graph T is easily seen. □ 

5.4 Bounds on the dimensions 

By virtue of Lemma 1 1 71 the map E is bilipschitz in a neighbourhood of the points 
of B. Since bilipschitzian mappings leave the dimensions of sets invariant (Tj, 
we may consider the Hausdorff or Minkowski dimension of the non-wandering 
set NW(.F). 

In order to obtain bounds on the Hausdorff and Minkowski dimensions of the 
non-wandering set NW(F) we use another theorem from pj3]. For the applica¬ 
tion of this theorem it is sufficient to know the values of the Lipschitz constants 
FT, Ux_ etc. and the constants Ci and di. 

Theorem 4 ( [TP] , Theorem 5). Suppose that the Lipschitz dynamical system 
F : B —> B possesses a pre-Markov partition B = Li(L 1 B i with an oriented graph 
F. Suppose also that the map F is invertible, and that 

16 The condition below actually guarantees the invertibility of F. 

17 According to Definition [T] 
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u x lJ Vy lJ — Uy l3 v x L3 > 0 for each directed edge (ij) of the graph T, 

and that F and F^ 1 satisfy the conditions of Lipschitz hyperbolicity with con¬ 
stants Ci and di, respectively, and that Cidi < 1 for 1 < i < N. If the graph T is 
strongly connected, then 

(s'_ + s' + )a < dim# NW(F) < dim#NW(.F) < s_ + s+. 

The constants s_, s'_, s+, and s', are chosen such that the spectral radius of 
the corresponding matrix equals 1, 

p(( A-)»_) = 1, p{( A-) S '_) = 1, p((A+) a+ ) = 1, p((A' + )s' + ) = X 

where A± = (Xfj), A' ± = (A f^), an$f\ A s = (afy) for A = a^- > 0, and 

a = min CT {min{a“, aj}}, where the minimum is over all elementary cycles cr 
in the graph T. The values of off are defined in m and m- 

Here for (ij) a directed edge of T the numbers A~ and A“ are given by the 
formulas m and m, and A^ and Aj) by the fonnula S20\) . If (ij) is not an 
edge of T, then A“- = A"-' = A+, = At = 0. 

The numerical values of the constants s±, s(j_, and a for the map F under 
consideration have been calculated with the aid of a computer and are given by 

s + = 0.408; s' + = 0.327; s_ = 0.876; s'_ = 0.593. (21) 

Note that solving the equation p(A s ) = 1 with respect to s seems to be a 
nontrivial task at the first glance. However, the following result on the mono¬ 
tonicity of the function p(A s ) with respect to s guarantees the existence and 
uniqueness of the solution and indicates a simple numerical solution method. 

Proposition 1 ( [10]). Let A = (A ij) be an irreducible N x N matrix with 
nonnegative coefficients. If the coefficients Xj satisfy condition on the 

contraction along elementary cycles, then the function p(A s ) is continuous and 
strictly monotonically decreasing with s. Moreover, p( Aq) > 1 and p(A s ) — > 0 
as s ^ Too. 

The numerical values of the coefficients off for F and F ~ 1 along all elemen¬ 
tary cycles are given in Tables [3] and 0] respectively. Therefore 

a = min {min{a~, at }} = 0.625 . 

Lemma 21. The Hausdorff and Minkowski dimensions of the non-wandering 
set NW (F) satisfy the bounds 

0.575 < dim# NW(F) < dh^#NW(F) < 1.284 . (22) 

ls The matrix A s is obtained by taking element-wise the s-th power of the nonnegative 
matrix A. If s = 0, then Ao is a (0,l)-matrix with ones indicating the non-zero elements of 
the original matrix A. 
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5.5 Unilateral Markov chain 


In the second part of this paper we will study general Hamiltonian systems with 
discontinuous right-hand side. For this we will need the description of the set 
of points in the quotient with respect to the action g which are not attracted 
to the semi-singular cycles Z-ij in the forward time direction, regardless of the 
asymptotic behaviour in the backward time direction. This will be the subject 
of this subsection. 

Obviously the points of the intersection Q k (~| D 3 \ , k = 1,3, are not at¬ 
tracted to Z 13 when evolving in the forward time direction under iterations of 
the Poincare map because by virtue of Lemma [TUI we have (ij)$ 2 (Q k (iDij) = 
Q k fl Dij , k = i,j. Moreover, in El it was shown that the differential of the 
Poincare map in the ambient quotient space, <f> : S/g S/g 7 has exactly one 
eigenvalue with modulus strictly smaller than 1. By virtue of the Hadamard- 
Perron theorem [3] on S/g there exists a smooth immersed one-dimensional 
stable submanifold Q k , consisting of points which are attracted to Q k fl D,j , 
k = i 7 j, under the iterations of the map <b. If we pass to the extended phase 
space Ai = T*M we obtain that the trajectories of Hamiltonian system m 
emanating from the preimage n 1 (Q k ) hit the origin in finite time and are thus 
optimal by Theorem [T| Hence on D 3 i there exist two one-dimensional Lips- 
chitJ^l immersed submanifolds Q k D Q k fl -D 13 , k = 1,3, such that for every 
point z € Q k the images $ n (z) are not attracted to the semi-singular cycles Z i: j 
as n—> 00 . 

Besides Q k 7 k = 1,3, on D 3i there exists a set y of points such that their 
images do not tend to Z i? neither. The set y is fractal, it consists of the union of 
sets where each ik denotes one of the symbols A or C. The structure 

of these sets is described in Lemma 4 of E- Each such set is a Lipschitz 
curve from the functions space Lip dj (Y,- —> Xj) on the corresponding rectangle 

Bj D iQ.ii. Consider the natural map T- from y to the unilateral Markov chain 
X~ of paths on the graph T which are infinite to the right. The preimage of a 
point in E~ is the corresponding Lipschitz curve It is clear that by 

construction the map semi-conjugates the left shift l on £~ with the map F, 
i.e., l o Tp = o F. The map is continuous by virtue of Remark 1 of [ID] . 
Bounds on the dimension of the set y have also been calculated in noB 

1 + a+s+ < dim// y < dinisJ^ < 1 + s+, 
where a+ = min^ log 'V, . 

log a z 

19 There is no smooth structure on I)?. 1 which is preserved by the map E. Therefore we 
consider only the Lipschitzian structure on D 31 , which is left invariant by the map E , see 
Theorem [T] and Lemma 1 171 

20 Unfortunately, in QQI this bound has not been singled out into a separate proposition. 
For details we refer the reader to the proof of Theorem 5. More precisely, above bounds on 
the dimensions dim# y and dim#T are exactly the bounds on the dimensions of the sets 
in m, which have been obtained in the proof of Theorem 5. 
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Figure 21: The graph T. The letters r and t are irrelevant for the definition 
of the graph T. They are necessary for the construction of the graph T in 
Definition [HI 


We have proven the following lemma. 

Lemma 22 . 1. On the disc D 31 there exists a one-dimensional immersed 

Lipschitz submanifold Q 2 , consisting of points which under the iterations 
of the Poincare map $ tend to the intersection points of the four-link cycle 
Q 2 with S/g. 

2. On the disc D 31 there exists a set y C B with the following properties, (i) 
For every point z € y the images F n (z) do not tend to the semi-singular 
cycles Zij as n — > +00. (ii) The restriction of F ony is by the continuous 
map T- semi-conjugated to the left shift on the unilateral Markov chain 
E~ . Here the preimage of any point o/E~ is a one-dimensional Lipschitz 
curve in B. (Hi) The dimensions of the set y satisfy the bounds 

1.204 < dim// y < dim#}’ < 1.408. 


5.6 The fractal nature of the Poincare map 

The map F plays of course an auxiliary role. Our main goal is the investigation 
of the Poincare return map *I>. In order to formulate the main theorem on the 
map $ we shall need the following graph T. 

Definition 8. The graph T depicted on Fig. [711 is a prototype of the directed 
graph r. The graph F can be obtain from T as follows. The vertex set of T is 
a direct product of the vertex set of T with the set of ordered pairs (ij), where 
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i,j £ { 1 , 2 , 3}, (* 7 ^ j). Hence the graph T has 24 vertices A 12 , A 33 , ...D 32 . In T 
there exists a directed edge from the vertex Ajj to the vertex B,'ji if and only 
if j = i' and i ^ j', i.e., {i,j = i !,/} = {1, 2, 3}. This rule applies to all edges 
in T which are marked by the letter r. There exists a directed edge from the 
vertex Aij to the vertex GVy if and only if i' = j and j' = i. This rule applies 
to all edges in T which are marked by the letter t. There are no other edges in 

r. 


Note that the graph T consists of two disjoint and mutually isomorphic 
components T±, which are strongly connected and each possess one component 
in the spectral decompositiorf^l. 

Theorem 5. The dynamics of the Poincare map $ : (SC\M + )/g —> ( SC\M + )/g 
can be described as follows: 

1. The non-wandering sety = NW($) C (SC\M + )/g of the map <f> is home- 
omorphic to the topological chain Er of bilaterally infinite paths on the 
graph T. The homeomorphism is realized by a mapping Tp which conju¬ 
gates 4> and the left shift l on £r> the following diagram commutes: 


•1'I'r 1'4'r 

E r — E r 

The dimensions of the set y satisfy the bounds 

0.575 < dim/fj^ < dini/j^ < 1.284. 

2. The set of points z £ (S fl M + )/g which are not attracted to one of the 
semi-singular cycles Zij under the iterations of <f> contains (i) the stable 
manifolds Qf of $ emanating from the hyperbolic points Q l fl ( S/g); (ii) 
a set y + C (5fl M + )/g which is invariant under i.e., 4>(3^ + ) C y + . 
The dynamics of <f> on y + is semi-conjugated to the left shift l on the 
topological Markov chain Ejt of paths on the graph T which are infinite to 
the right. The conjugation is realized by some map Tp : y + —> Ep, i.e., 
the following diagram commutes: 

j;+ -?-► 3;+ 

Ejt —^—> Ejt 

21 A graph possesses one component in the spectral decomposition if the greatest common 
divisor of the lengths of its elementary cycles equals 1 [4]. 
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The map H/p is continuous, surjective, and the preimage of any point is a 
Lipschitz submanifold homeomorphic to a line segment. The dimensions 
of the set y + satisfy the bounds 

1.204 < dim# y + < < 1-408. 

Note that item (ii) of Theorem [5] does not describe all points 2 € (SC\M + )/g 
which do not tend to one of the semi-singular cycles Z. l: j under the iteration of 
the map <b. For instance, Theorem [5] does not mention the three-link cycles. 

Proof of Theorem 0 We start with the proof of item 1. Denote by ^(d*) the set 
of points z £ S/g such that the images do not tend to any of the three 

semi-singular cycles in the forward time direction, i.e., as k —> +oo, and do 
not tend to any of the two three-link cycles Z ± in backward time direction, i.e., 
as k —> — oo. 

In order to describe the set S($) and the dynamical system $ : 5'( < i>) —»• S($) 
we consider the intersection of the set S(3>) with the disc D 3 i. Since the map 
F is given by 4> 2 on A. A U C.A and by <b 3 on A.C U C.C, the set S(F) contains 
the sets NW(F), $[NW(F)], and $ 2 [NW(F)]. Applying all permutations from 
S 3 to NW(F) we obtain that 

S($) D 5 i($) = |J ct(W(F)U$[NW(F)] U$ 2 [NW(F)]). 

o-eS3 

In Subsections 14.91 and 14.101 it has been shown that the intersection of the 
difference S(3>) \ <Si($) with the closure of region III on the disc D 3 \ consists 
of points z corresponding to a sequence (... i-\i 3 .i \...), ik € {A, B , C}, ending 
with an infinite tail of symbols B only. Any such point z lies on the stable 
manifold Q 2 and its images under $ tend to the four-link cycle Q 2 in the forward 
direction of time. The other points of the intersection (S , ($) \ Si(3>)) D D 33 can 
be obtained by application of or <h 2 to the points z described previously and 
a corresponding permutation in S 3 . Hence we obtain 

s , ($)\s 1 ($) = s($) C |Jg\ 

i 

and therefore the points of the set S($>) are not non-wandering points of the map 
$. The points in the set (S/g) \ S'( < i>) are also not non-wandering points of <h, 
since they are attracted to one of the three-link cycles Z ± under the iterations 
of d)” 1 , or are attracted to some point on one of the semi-singular segments Zf 3 
under the iterations of $. Therefore 

NW($) = Si ($) = |_| (j (NW(F) U $ (NW(F)) U $ 2 (NW(F))). 

<t es 3 

Taking into account the results of Lemmas [20] and [21] it remains to explain 
how the graph T arises and to construct the map 4/p in order to complete the 
proof of item 1. 
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First note that the topological Markov chain Ep is identical to the chain E ac 
of bilaterally infinite words with the two-letter alphabet {A, C}. Recall that up 
to the application of a permutation returning the point to the disc £>31 we have 
the identity F = $ 3 on the set A.CUC.C and the identity F = >f > 2 on A.AUC.A. 
Let us consider the map cr(®(z)) on the set NW(<I>) fl£> 3 i, where er G S 3 

takes to £> 31 - Then the map Ty from Lemma EUl naturally gives rise to 
a homeomorphisni Tp conjugating the dynamical system T : NW(<f>) fl £>31 —> 
NW(<f>) fl £>31 with the left shift on the topological Markov chain Ep. Here 
the directed graph T has four vertices and is depicted on Fig. [21] In order to 
return to the original map $ we have to take into account all discs £> l? and to 
track in which of the six discs each image <I> fc ( 2 ) is located. This can be easily 
accomplished by utilizing the letters r and t which label the edges of the graph 
on Fig. [5T] (recall that A = rr and C = trt). For the disc £>31 the letter r 
corresponds to the permutation (312) G S 3 , and the letter t to the transposition 
(13) G S 3 . This finally yields that the restriction of the map $ to the non¬ 
wandering set NW($) is conjugated to the left shift on the chain Ep. The first 
part of the theorem has been proven. 

Item (i) of the second part of the theorem is trivial. The proof of item (ii) 
can be based on Lemma [22] and is identical to the proof of the first part, given 
we set, e.g., 

(y + = |_| u$ 2 ( 5 >)). 


Remark 7. The set y + can be chosen in many ways. One may replace it, e.g., 
by $ fe (3^ + ) for any k G Z. However, working with the union (J kG % ^ (J^ + ) 
encounters some difficulties. Firstly, its Minkowski dimension may be strictly 
greater than the Minkowski dimension of y + . Moreover, the preimage of any 
point of Ep in Ufcez < f >fe (3^ + ) is no more a Lipschitz submanifold, but rather an 
immersed Lipschitz submanifold. This second difficulty is similar to the mingling 
of the stable and unstable manifolds of some fixed point of a hyperbolic map if 
we move farther away from the fixed point. 

5.7 Main theorem on the chaotic nature of the optimal 
synthesis in the model problem 

We are now ready to formulate the main theorem of the first part of this paper, 
describing the dynamics of the optimal trajectories in the original optimal con¬ 
trol problem ©• The first three items of the theorem describe the considered 
sets X and X + , consisting of trajectories of Hamiltonian system J3]), and the 
remaining items describe the chaotic dynamics on these sets. Since every trajec¬ 
tory on the sets X and X + intersects the stratified manifold S = S 12 U S 13 U S 23 
of discontinuity of the right-hand side of system m , the chaotic dynamics of 
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these trajectories can best be described in terms of the sequence of intersections 
of the strata <Sy . 

Theorem 6 . In the extended phase space A4 = T*M of problem m there exist 
two sets X and X + with the following properties. 

(I) For every point z £ X U X + there exists a time T(z) < oo such that 
the trajectory X(t,z) of Hamiltonian system [3\) through z exists and is 
unique for all t £ (—oo,T(z)]. The trajectory X(t,z) hits the origin at 
time T(z), X(T(z), z) = 0. 

(II) The sets X and X + are invariant with respect to the flow of Hamiltonian 
system m in the following sense. If z £ X + , then X(t, z) lies in X + for 
all t £ [0, T(z)). If z £ X, then X(t, z) lies in X for all t £ (—oo, T(z)). 

(III) The projection of the trajectory X(t, z) on phase space M, extended by 0 
for t > T(z), is optimal for all z £ X U X + , i.e., X U X + C M + . More¬ 
over, the trajectory X(t,z) intersects the switching surface S a countable 
number of times at time instants ... < t_i < to < 0 < t\ < t? ... < T(z), 
X(tk,z) £ S, where tk —> T(z ) as k +oo and tk —> —oo as k —»• —oo. 

(IV) The dynamical system $ : <Tn5 —> XnS which takes a point z £ X fl S to 
the next intersection point of the trajectory X(t, z) with the switching sur¬ 
face S, < f > (z) = X(ti,z), is semi-conjugated with the topological Markov 
chain of bilatei'ally infinite paths on the graph T by means of some map 

1 ': 


xn s^xns 

E r ---> E r 

The map ^p is continuous and surjective, and the preimage of any point 
a £ Ep is a one-dimensional smooth manifold, diffeomorphic to an open 
interval. Denote by .V the set of all paths on T which commence at 
some vertexV. Then the preimages [-Ay], tltp [.By], t&p [.Cy], and 
'E'p 1 [.Dy] lie on the stratum Sij. 

(V) The dynamical system $ : X + fl S —> X + fl S which takes a point z £ 
X+ fi S to the next intersection point of the trajectory X(t,z) with the 
switching surface S, ^(z) = X(t\,z), is semi-conjugated with the left 
shift on the topological Markov chain Ep of paths on T which are infinite 
to the right (see Definition 0). The conjugation is realized by some map 
'fp, i.e., the following diagram is commutative: 

x+ns — x + ns 

w , w 

Ep ---> Ep 
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The map 'Fp is continuous and surjective, and the preimage of any point 
a £ Ep is a Lipschitz submanifold, whose relative interior is homeo- 
morphic to a two-dimensional open disc. The preimages (\Fp) —1 [.j4y], 
('E'p )~ 1 [.Bij], (\Fp ) _1 [.Cp,], and (\Fp) _1 [..Dy] lie on the stratum Sij. 

(VI) The Hausdorff and Minkowski dimensions of X and X + satisfy the bounds 


3.204 < dim B X + < dim B X + < 3.408, 

2.575 < dim g X < dm( B X < 3.284. 

(VII) The topological entropy of the left shift l on Ep and Ep equals 


h\,op (l) 


log 2 




1 

2 ~~ "IfT 


0.4057. 


(VIII) Similar assertions hold for trajectories of Hamiltonian system (H which 
leave the origin, except that they are no more optimal in problem (Hl- 


Proof. In order to construct the sets X and X + we use the sets y and y + , re¬ 
spectively, which have been constructed in Theorem[5j There are two differences 
between these pairs of sets. Firstly, the latter pair lies in quotient space M + /g, 
while the former lies in extended phase space M. Secondly, the sets y and y + 
lie on the switching surface S/g and do not contain the optimal trajectories it¬ 
self. Let us first construct intermediate sets Z and Z + . For every point in y or 
y + we place the projection of the trajectory of Hamiltonian system ([3]) through 
this point into the set Z or Z + , respectively. The set Z shall contain the whole 
trajectory, and the set Z+ only half of it, corresponding to the forward time 
direction. Note that Z D (S/g) = y and Z + D (S/g) = y + , since the sets y and 
jV + are invariant with respect to the map <F, and therefore no new points on S 
will appear. The sets X and X + are then simply defined as the preimages of Z 
and Z + under the projection tt : M —> M/g. Since the sets y and y + lie in 
M + /g , we have that every trajectory through a point in n~ 1 (y) or 7 t _1 (3^ + ) is 
optimal and reaches the origin in finite time. This proves items (I-III). 

The semi-conjugating maps Tr and 'Fp are both constructed by the same 
procedure. First we apply the projection 7r onto the factor space M./g , and 
then the corresponding map Tr or Tp from Theorem [5j 

Tp = ° tt and Tjt = IFp o tt. 

Let us first study the properties of the map Tp. Clearly it is continuous 
as a composition of continuous maps. It is surjective as a composition of a 
surjective and a bijective map. In order to avoid confusion, we shall now denote 
the Poincare map on the original switching surface S by <f>^ and on the quotient 
S/g by 4>f. Then the commutativity of the diagram from item (IV) follows from 
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’Fp o $1 = Tp O 7T O = tUp O <jH O 7T = l O \Fp O TT = l O Tp. 

Here the second relation holds because Hamiltonian system © respects the 
action g , and hence this action is respected by the Poincare map. The third 
relation holds by virtue of Theorem [5] The preimage 7r _1 (z) of any point z £ 
M + /g is given by a smooth one-dinrensional curve, namely the orbit of the 
action g of the group R + . Since the map tbp is bijective, we have that the 
preimage ’Fj7 1 (<7) is a smooth curve for every a £ Sp. This proves item (IV). 

The proof of item (V) for the mapping *Pp is almost identical to the proof 
of item (IV). The only difference appears in the consideration of the preimage 
(\Hp) —1 (cr) of points a £ Sp. In Theorem [5] it was shown that ( , Fp) _1 (cr) is 
a Lipscliitz curve in (S fi M + )/g. By virtue of the structure of the projec¬ 
tion 7 r _1 we obtain that (’Pp) _1 (cr) is a two-dimensional Lipschitz submanifold, 
homeomorphic to the direct product of a segment and an interval. 

In order to obtain the bounds in item (VI) we shall show that 


dim# X > 1 + dim# Z > 2 + dim# V; dim#< ; f < 1 + dim#Z < 2 + dim#V; 

dim# X + > l+dim#Z + > 2+dim#V + ; dim n X + < l+dim#Z + < 2+dim#V + . 

These bounds are obtained by application of the formulas for the Hausdorff and 
Minkowski dimensions of direct products of sets, see [I], Chapter 7. 

Let us now compute the topological entropies of the left shifts on Sp and 
Sp. Since the graph T is not connected and consists of two disjoint copies 
of some graph T', we may replace the graph T by T' for the calculus of the 
topological entropy. The graph T' can be obtained from T by the identification 
of the vertices Aij with Aji , B tJ with Bji etc. Denote by Hoi the 0,1-matrix 
generated by the graph T'. The matrix A 0 1 is irreducible, because T' is strongly 
connected and the greatest common divisor of the length of its cycles is 1. 
By virtue of the Perron-Frobenius theorem the matrix Hoi then has a unique 
positive eigenvalue A max > 0. The sought topological entropies then coincide 
and equal log 2 (A max ) g]. 

Let p be the eigenvector of Hoi corresponding to the eigenvalue A max . Since 
both A max and p are unique (up to a homothety), p must be invariant with 
respect to all coordinate permutations which are induced by the action of the 
group S 3 on the graph I '. Hence if the vertex M 12 corresponds to an entry a 
of the vector p , then all other vertices of the form Aij also correspond to an 
entry a in p. The vertices £>,; 7 and C, 7 then correspond to the entry A max a, 
and the vertices 2Ay to the entry A^, ax a. We then obtain the algebraic relation 
a = A^ ax a + Af nax a. Therefore 

\3 1 \2 _ 1 _ n 

/v max ' /v max ^ 

The value of A max claimed in the theorem can then be obtained by the Cardano 
formula. 
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Figure 22: Arrangement of the strata Sf^ 3 and the domains fli 


The last item of the theorem is proven by application of the inversion g(— 1) 
to Hamiltonian system ©. □ 

Remark 8. The Hausdorff and Minkowski dimensions of the sets X and X + do 
not change if we project the sets on the phase space M = {(a;, y )} = R 4 , because 
the map E is locally Lipschitz and hence the projection M + = graph E —> M is 
locally bilipschitz. 

Remark 9. The set S defined in Theorem [2] for the case of an equilateral triangle 
H is a subset of X. 


6 Chaos on finite time intervals in Hamiltonian 
systems with discontinuous right-hand side 

In this section we prove and formulate two theorems on the chaotic behaviour 
of trajectories in integral vortices in Hamiltonian systems with discontinuous 
right-hand side in generic position. The proofs of these theorems are based 
on the structure of optimal synthesis in model problem 0i more precisely on 
Theorems [2] and [6] 

6.1 Hamiltonian systems with discontinuous right-hand 
side 

Consider a smooth symplectic manifold At 2 ” of dimension 2 n. Suppose that 
a (2 n — l)-dimensional stratified submanifold S H C At partitions At into a 
finite number of open domains fli,..., flk, (At = (Jfb)- Consider a smooth 
Hamiltonian H(q,p) : At — > R such that its restriction Hi = H\n i to any of the 
domains fli is a smooth function, which is C^-extendable to a neighbourhood 
of the closure fConsider an open set U C At. Suppose that the set U contains 
parts of only three (2n — 1 (-dimensional strata C S H , (■ i,j = 1,2,3), such 
that separates the domains fli and flj. Let the submanifolds Sfj join in a 
stratum S ^ 23 of dimension (2 n — 2), as depicted on Fig. [22] 

The main tool in the study of the generic behaviour of solutions of differ¬ 
ential equations is the investigation of their typical singularities. For systems 
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of ordinary differential equations with continuous right-hand side this program 
was at large realized in the work of Poincare. But in optimal control theory 
the key role is played by Hamiltonian systems with discontinuous right-hand 
side and a tangential jump in the velocity vector, which are generated by the 
application of the Pontryagin Maximum Principle. 

The solutions of piece-wise smooth Hamiltonian systems which enter into a 
singular point xq of order 2, lying on a switching surface S of codimension 1, have 
been studied in the work of I. Kupka and Zelikin & Borisov 0 , m ■ There it 
was shown that while approaching the point Xq, these solutions experience a bi¬ 
constant ratio chattering in generic position. This means that they intersect the 
switching surface S at time instants t\, t 2 , • • •, t„,... forming an asymptotically 
bi-constant ratio sequence, 



hm -= a, inn - 

n->oo t2n+l ~ t 2n n->oo t 2n +2 _ ^2n+l 


In this section we study the situation when the singular point Xq of order 2 
lies on a stratum of codimension 2. In this case a qualitatively new phenomenon 
arises, which is typical for problems of optimal synthesis and for Hamiltonian 
systems with discontinuous right-hand side. Besides the trajectories entering 
into xq with usual chattering there exists a Cantor set Xh(xq) of trajectories, 
the dynamics of whose intersection with the strata can be described by a 
topological Markov chain Ejt. Here T is a certain oriented graph (see Def. [SJ). 
More precisely, Ejt is a quotient of the system describing the dynamics of the 
intersection of the trajectories in Xh{xo) with . The intersection of the set 
X h (xq) of trajectories with a surface transversal to them is a multi-dimensional 
Cantor set similar to Smale’s Horseshoe. The topological Markov chain Ep is 
homeomorphic to the standard version of Smale’s Horseshoe [3] as a topological 
space, but not conjugated to it as a dynamical system. 

6.2 Formulation of theorems on chaos in Hamiltonian sys¬ 
tems with discontinuous right-hand side 

Before formulating the main theorems, we shall formally define the configuration 
depicted on Fig. [22] We shall demand that the following inequalities be satisfied, 
where H may be replaced by — H: 


Hi{x) > m&x(Hj(x), Hk{x)) Vx G f2j, 
for all mutually different i, j, and k from {1,2,3}. Set 


(23) 


3 G = 3 F 0 = H 1 + H 2 + H 3 , 

3F\ = H 2 - H 3 , 3F 2 = H 3 - H u 3 F 3 = H 1 - H 2 . 


(24) 


Clearly the Ft, are uniquely determined by G, i 7 }, and F 2 and vice versa. Here 
F 3 = —Fi — F 2 has been introduced for brevity of notations. 
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Definition 9. A point xq £ <S^ 3 , Hi(xq) = H 2 (xq) = iJ 3 ( xo) will be called 
strange©! if the following conditions are satisfied at Xq\ 

(i) In some neighbourhood of xq condition (E3l) is satisfied, where H is possibly 
replaced by — H. 

(ii) The functions F r , (ad Fi)F r , (adFj)(adFj)F r , and ( ad Fk) (ad Fj) (ad Fi)F r 
vanish at xq. Here the index r runs through the set {1, 2, 3}, and the indices 
i,j , k through {0,1, 2,31. The set of the differentials of these functions at 
Xo has the maximal ranlJ 23 l allowed by the anti-commutativity and linearity 
relations, the Jacobi identities, and the relation F\ +F 2 + F 3 = 0. In other 
words, these differentials are in general position. 

(iii) The bilinear form 

B rr i = adi 7 ’ r (adG) 3 F r ./| Xo , r,r' = 1,2,3 

has maximal rank 2. Moreover, it is symmetric and negative semi-definite 
if condition ESI) holds for H 1 and positive semi-definite if this condition 
holds for —H. All other commutators of 5-th order of the functions G = Fq 
and F r , r = 1, 2,3 which are independent of the aforementioned ones vanish 
at the point Xq. 

The behaviour of the trajectories in the integral vortex of trajectories joining 
.To is described in the two Theorems[7]and[5]below. In the first of these theorems 
a much more detailed description of the chaotic behaviour of the trajectories 
is given than in the second one, but the bilinear form B rr i has to satisfy an 
additional requirement, which for the model problem © is equivalent to the 
condition that the triangle of controls is equilateral, as in Theorem [SI In the 
second one of these theorems this condition is replaced by the condition of 
closeness to an equilateral triangle, as in Theorem [2] 

Theorem 7. Suppose the point Xq of a Hamiltonian system with piece-wise 
smooth Hamiltonian H is strange, as in Definition [3 and the form B rr i is 
proportional to the bilinear form given by the matrix 

( -1 1/2 1/2 \ 

1/2 -1 1/2 (25) 

V 1/2 1/2 -1 J 

with a positive proportionality factor if condition HHi) is satisfied for H, and 
with negative proportionality factor if condition S23\) is satisfied for —H. 

Then there exists a set Xh{%o) of points in the neighbourhood of xq such 
that 

22 We hope that this notation will be associated by the reader with strange attractors. 

23 This conditions may be weakened. 
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(I) For every point z £ Xh(xq) there exists a time T(z) < oo such that 
the trajectory X(t,z) through z exists and is unique for all t £ [0;T(2)]. 
Moreover, the trajectory X(t,z) hits xp at time T(z), X(T(z), z) = Xq. 

(II) The set Xjj(x o) consists of trajectories of the Hamiltonian system with 
Hamiltonian H and is invariant with respect to this system in the follow¬ 
ing sense: if z £ Xh(x o), thenX{t,z) lies in Xh[xq) for allt£ [ 0 ; T ( z )). 
Moreover, in the interval t £ (0;T(^)) the trajectory X(t,z) intersects 
the switching surface S H an infinite, countable number of times at the 
moments t\ < t 2 < ■ ■ X(tk, z) £ S H , where tk —> T(z) as k +oo. 

(Ill) The dynamical system <&h '■ Xh(x o) (1 S h —> X b (xq) D S h which takes 
the point z £ X B (x o) C\S h to the next intersection point of the trajectory 
X(t,z) with S H , i.e., &h(z) = X(ti,z), is by means of a map typ semi- 
conjugated to the unilateral topological Markov chain £p of paths on the 
fixed graph T (see Definition [3^ which are infinite to the right, where T is 
independent of Xq and H, 


x H (x 0 ) n s H x H (xo) n s H 

l*¥ . I*? 

£+ -- > £+ 

Here l is the Bernoulli left, shift, and is a continuous surjective map. 
The preimage ('E'f f ) _1 (cr) of any point a £ £p is homeomorphic to an 
open unit disc D 2 , and the diameter 0 /tends to 0 as 
k —> + 00 . Denote by (. V ) the set of paths in T which start at the 
vertex V. Then the preimages of the sets ('frp)~ 1 (.Aij), (fi>p)~ 1 (.Bij), 
('I l p)~ 1 (.Cij), and (fi r p)~ 1 (.Dij) lie on the stratum Sj? with the same 
indices. 

(IV) /FI dG(x 0 ) = 0, then the Hausdorff and Minkowski dimensions of the 
set Xh(x 0 ) do not depend on xo and H (of course, the point xq has to 
satisfy the conditions of the theorem) and coincide with the dimensions 
of the set X given in Theorem [31 Hence the bounds 


3.204762 < dim h X H (x 0 ) < dim B X H (x 0 ) < 3.407495 (26) 

are valid for these dimensions. 

(V) The topological entropy of the Bernoulli shift l is given by 


htop(l) — lo§2 



0.4057. 


24 Unfortunately an error has slipped into the papers mm and the condition dG(x o) = 0 
has been forgotten. 
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(VI) Similar statements are valid for the trajectories leaving the point xq if the 
direction of time flow is reversed. 


The set Xh{x o) is the direct analogue of the set X + from Theorem [G] For¬ 
mally Hamiltonian system (JoJ of model problem ([1]) does not satisfy the con¬ 
ditions of Theorem ©, however, because most of the Poisson brackets in the 
definition of the strange point vanish identically. Therefore the condition of 
linear independence of their gradients at the point does not hold. We could 
formulate Theorem [7] in a way such that these cases satisfy its conditions, but 
then we would lose the structural stability of the phenomenon, see Remark ITOl) . 

Remark 10. Using methods for the computation of the topological entropy it is 
not hard to calculate the Perry measure p,p on Ep which satisfies the variational 
principle and maximizes the metric entropy: 

nia.x hfi (/ j — h^ipif) — h± 0 p{lf 
A 4 

Theorem 8. Let H be a piece-wise smooth Hamiltonian and Xq be a strange 
point of the corresponding of a Hamiltonian system in the sense of Definition 
03 Suppose there exists a number Xh{xo), positive if condition 1231) holds for H 
and negative if condition \23 1) holds for —H, such that the form \H{xo)B rr i is 
close enouan 25 \ to the bilinear form with matrix V25\) . 

Then in the neighbourhood of Xq there exists a set of points £ 0 ) with the 

following properties: 

(I) For every point z £ Ep(xo) there exists a time T(z) < 00 such that 
the trajectory X ( t , z) passing through z exists and is unique for t £ 
[0,T(»]. Moreover, the trajectory X(t,z) hits the point xo at time T{z), 
X(T(z),z) = x 0 - 

(II) The set Eh(xq) consists of trajectories of the Hamiltonian system with 
Hamiltonian H and is invariant with respect to this system in the follow¬ 
ing sense. If z £ Eh{xq), thenX(t,z) lies in Eh (xo) for all t £ [0,T(2:)). 
Moreover, in the time interval t £ (0 ,T(z)) the trajectory X(t,z) inter¬ 
sects the switching surface S H a countable, infinite number of times at 
time instants t\ < £2 < • ■ •. X(tk,z) £ S H , where tk —> T(z) as k —> +00. 

(Ill) Consider the dynamical system : cs H —> ^h(xo)L\S h taking 

a point z £ Eh(x 0 ) on S H to the next intersection point of the trajectory 
X(t,z) with S H , i.e., &h(z) = X(ti,z). There exists a natural number 
n > 0, independent of the Hamiltonian system and the point xo and 
coinciding with the number n from m, such that the map is semi- 
conjugated with the the topological Markov chain of the Bernoulli shift on 
the disjoint union of two copies of the space E^ of unilaterally infinite 

2a Here ’’close enough” means that there exists a neighbourhood of matrix in the man¬ 
ifold of symmetric matrices of rank 2, independent of xq and H , and a number \h(%o) such 
that the form \h( x q)B rr / is an element of this neighbourhood. 
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words with letters 0 and 1. In other words, there exists a continuous 

2 

surjective map from S#(xo) D S H to the space U ^oi such that the 
following diagram commutes: 


SffOr o) n S H — ^ Zh{x o) n S H 

2 i< , 2 K 

U s oi - >uvti 


Here l denotes the left shift on each of the copies of S^. The preimage 
(*oi)~V) °f every point a e E+ is homeomorphic to the open two- 
dimensional disc D 2 , and the diameter of (('I'oi)~ 1 (' T )) tends to 0 as 
k —> +oo. 

(IV) Besides the Cantor set 'Bh(x o) of trajectories which has been described in 
(I)-(IV) above, there are two other sets of chattering trajectories which 
enter into the point Xq, namely 

(a) There exist two 1-parametric families of "three-chain’' trajectories 
i?i 23 and i?i 32 which hit the point Xo in finite time with chattering, 
i.e., with a countable number of consecutive intersections of the strata 
S( 2 , S 23 , and S |L The strata are intersected in the described order 
for R 123 and in the reverse order for R 132 ■ 

(b) There exist three 2-parametric families of ’’four-chain” trajectories 
Q 1 , Q 2 , and Q 3 . Each trajectory from Qi intersects consecutively 
and cyclically the strata Sfj, S((, S((, and Sfj a countable number 
of times, where i 7 ^ j ^ k. 

(V) Similar statements are valid for the trajectories leaving the point xq if the 
direction of time flow is reversed. 

Remark 11. If the condition for an equilateral triangle from Theorem [7] is sat¬ 
isfied, then the set Eh{x 0 ) is a subset of Xh(x 0 ). 

6.3 Descending system of Poisson brackets 

The proofs of Theorems [7] and [ 8 ] are similar and based on key facts about the 
chaotic behaviour of optimal synthesis in the model problem, which have been 
obtained in the proofs of Theorems [G] and El respectively. We wish to find 
the optimal synthesis of model problem m, which was described in Theorems 
[7] and [ 8 ] and which sits in a generic Hamiltonian system with discontinuous 
right-hand side. To this end we use the so-called descending system of Poisson 
brackets. This allows us to perform an effective investigation of the behaviour of 
trajectories in integral vortices. For example, the theorem on the Hamiltonian 
nature of the flow of singular trajectories (see m or the theorem on the structure 
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of the Lagrangian manifold in the neighbourhood of a singular trajectory of first 
order for extremal problems which are holonomous with respect to a control 
taking values in a polyhedron (see El), have been proven with the method of 
the descending system of Poisson brackets. 

Let us describe the structure of the descending system of Poisson brackets 
in detail. Instead of the Hamiltonians Hi we shall use the Hamiltonians Fi from 
(l24l) . Keep in mind that -Fi + F% + F 3 = 0. The Hamiltonian H can be written 
as follows, where we assume a summation over the set {1,2,3} over repeating 
upper and lower indices. 

3 

H = G + Y_l u r F r = G + u r F r , 

r =1 

where u £ R 3 , u 1 + u 2 + u 3 = 0, and the function u(x) is chosen as follows: if 
x £ Hfc and ( ijk ) is an even permutation, i.e., ( ijk ) £ A3 then 

u l (x) = —1 ,vP{x) = l 1 u k (x) =0. 

Thus u(x) is the corresponding vertex of the triangle 


O = conv{(—1,1, 0), (0, —1,1), (1,0, —1)} C U = {u £ R 3 : u\ + u 2 + u 3 = 0}. 

(27) 

Since the Hamiltonian H is not smooth, we define the trajectory of the 
Hamiltonian system of ODEs with discontinuous right-hand side according to 
Filippov [2], as follows. Let w be the symplectic form on M and i u the canonical 
isomorphism i UJ {z) : T*M —> T Z M defined by the form u>. Then an absolutely 
continuous curve z(t) is a trajectory of the Hamiltonian system of ODEs with 
Hamiltonian H if and only if for almost all t the differential inclusion 


z(t) £ conv{ accumulation points of i u dH(z) as 2 ^ z{t)} C T x{t) M 
holds. Equivalently, 


z(t) = ■i ul [dG{z(t)) + u r dF r (z(t))j , for some u £ a,rgma,x{u r F r (z(t))}. 

Remark 12. Note that we do not pre-define a scalar product on the space U = 
{u : u\ + U 2 + U 3 = 0} C R 3 . The sum u r F r is to be understood as the 
contraction of a vector and a co-vector, where u(x) £ U and F(x) £ U*. 

We now come to the construction of the descending system of Poisson brack¬ 
ets for the Hamiltonian H. To this end we single out a set of ODEs along an 
arbitrary trajectory of the system and partition it into rows. We shall write the 
symbol ~\k\ to indicate that the corresponding ODE belongs to row k. In the 
first row of the descending system we write the three ODEs 
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nr ^F i = {G,F i } + {F r ,F i }u r , i= 1,2,3. 

The second row contains two types of equations, 

I2r ±{G,F i } = {G,{G,F i }} + {F r ,{G,F i }}u r , i = 1,2,3; 
nr %{F i ,F j } = {G,{F i ,F i }} + {F r ,{F i ,F j }}u r , i,j = 1,2,3. 

In general, the m-th row contains the equations 

W j t JC m = {G,lC m } + {F r ,K; m }u r , 

where /C m = {K m , {K m ^ ... {K 2 , K{} .. .}}, K\ = F u i = 1,2,3, and the 
remaining symbols Kj denote G as well as Fj. The equations in row (m + 1) 
can be obtained by differentiating the right-hand side of the equations in the 
m-th row with respect to t. For example, we have 

]m + l\ = {G, {G,/C m }} + {F r ,{G,IC m }}u r 

at 

and 


lm+ir 4 {F^JC™} = {G^F^JC™}} + {F r ,{F t ,JC m }}u r , i = 1,2,3. 

at 

The descending system is written out up to the row with number 2h, where h 
is the order of the singularity at the point Xq (see [15]). In our case we have 
h = 2, since the control explicitly appears at Xq when differentiating the fourth 
time. 

Therefore in the m-th row we have equations on the time derivative of brack¬ 
ets of order m, and the right-hand sides of these equations are affine in u with 
brackets of order m + 1 as coefficients. 

We remark that the construction of the descending system is similar to the 
derivation of singular trajectories. In the latter case, the equalities Fi(x(t)) = 0 
are differentiated along the trajectory of the Hamiltonian system with Hamil¬ 
tonian Ft = G + F r u r . From our point of view the descending system, albeit it 
contains a large number of equations, is more favorable than direct differentia¬ 
tion, because it does not produce explicit derivatives of the control like u,il,.. 
which are usually inconvenient to work with. 

Definition 10. In the descending system we shall call the brackets F r , (ad G)F r , 
(adG) 2 F r , (adG) 3 F r , r = 1,2,3 the primary brackets. The other brackets of 
order not exceeding four shall be called secondar}0 

26 Note that the function G does not belong to the descending system and is hence neither 
a primary nor a secondary bracket. 
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The reduction of the Hamiltonian system with discontinuous right-hand side 
to the Hamiltonian system generated by the Pontryagin maximum principle 
when applied to model problem m is based on the fact that in the neighbour¬ 
hood of a strange point the secondary brackets in any given row to have a larger 
order of smallness than the primary brackets, and hence do not influence the 
qualitative behaviour of the system. 

Lemma 23. Consider an arbitrary trajectory x{t ) which enters a strange point 
xq = x(0) at t = T. Then there exists a constant c > 0 such that if K. m is a 
primary bracket of order to < 4 in the descending system, then for all t close to 
T we have 


\1C m (x(t)) \ < c(T — f) 5_m , 

and if /C m is a secondary Poisson bracket of order m < 4, then 

\K, m (x{t)) \ < c(T — f) 6_m . 

Proof. We proof the lemma by reverse induction over to. The main tool is 
the following formula, which is valid for every bracket /C m , to < 4 from the 
descending system for t € [0,T]: 

IC m = £ ({G, K. m }{x{r)) + u r {r){F r ,K, m }(x{T))) dr. (28) 

This follows because K. m (x(T)) = 0 for to < 4. 

Let us now prove the base of the induction. Consider the last row of the 
descending system. If /C 4 is a secondary bracket, then the integrand in 051) 
vanishes at t = T according to the definition of the strange point Xq . Hence for 
small T — t we have for some 64 > 0 that 

\IC 4 (x(t))\ < bi(T — t ) 2 , if /C 4 is a secondary bracket. 

Consider a primary bracket /C 4 in the last row of the descending system. For 
this bracket the integrand in (051) does not need to vanish at t = T. Hence the 
most which we can guarantee is 

\IC 4 (x(t))\ < b^{T — t), if /C 4 is a primary bracket. 

The induction step is very similar to the induction base. Suppose m < 4. If 
JC m is a secondary bracket, then the integrand in (051) contains only secondary 
brackets. Hence we obtain from the assumption of the induction for m + 1 that 
for some b m > 0 and for t < T close to T 

\JC m (x(t))\ < b m (T — t) 6 ~ m , if /C m is a secondary bracket. 

For a primary bracket /C m we obtain the following. The first summand in the 
integrand in (1281) is a primary bracket in row (to + 1 ), and the second summand 
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is a secondary bracket and is majorized by the first one for t < T close to T. 
Hence for small T — t > 0 we have 


\JC m (x(t))\ < b m (T - tf ~ m , if K m is a primary bracket. 

This completes the proof. □ 

Remark 13. At the end of the proofs of Theorems [7] and [5] we show that on the 
trajectories in the integral vortex of the strange point Xq which are of interest 
to us we have the following lower bound for the primary Poisson brackets: 


c"(T — t)< max 

m 




1 

5-m 


< c'(T - t) 


for some constants c'. e" > 0 . 

Of course bounds similar to those obtained in Lemma [23] are valid also for 
trajectories leaving the strange point. We have to replace T — t by the time 
which has elapsed since the trajectory left the strange point xq. 


6.4 Blow-up of the singularity at the strange point 

In this section we describe the blow-up of the singularity at the strange point 
in the case when the form B rr i is merely non-degenerate. We show that on the 
zero section, i.e., on the sphere which will be glued in in place of the strange 
point, we obtain a smooth sub-manifold which contains exactly the trajectories 
of Hamiltonian system © which is obtained by application of the PMP to model 
problem © with triangle H. Here the form B rr > generates the scalar product 
on the plane given by u 1 + u 2 + u 3 = 0 which defines the functional J in model 
problem ©. Whether the triangle Q is equilateral is hence determined by the 
form B rr /. 

Remark 14. In the sequel we shall assume without loss of generality that con¬ 
dition (l23l) holds for H. In this case the form B rr / is negative definite on U 
according to Definition [9l We then use the form —B rr : for the definition of a 
scalar product on U. 

Let us introduce new local coordinates in the neighbourhood of the point 
Xq. Consider the set of all possible brackets /C m , m < 4 from the descending 
system of Poisson brackets, all in all 255 bracket!^!. The set JC = {/C m , m < 
4} of functions is linearly dependent, because of the Jacobi identity, the anti¬ 
symmetry of the Poisson bracket, and due to the relation F\ + F 2 + -F 3 = 0. 
Moreover, some brackets vanish identically. However, according to Definition 
1 of strange point the gradients d/C m form a set of maximally possible rank at 
Xq. Therefore the set K defines a map of a neighbourhood of the point into 

27 The number 255 is obtained as follows. We have 3 brackets in the first row of the descend¬ 
ing system, namely Fi, F 2 , and F 3 . The bracket G does not participate in the differentiation 
in the construction of the descending system. There are 3 - 4 = 12 brackets in row two, namely 
{Fi, Fj} and {G, F{\. Continuing, we obtain 255 = 3 + 3- 4 + 3-4 2 + 3-4 3 in total. 
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a linear subspace of R 255 of dimensiort 23 ! 31. Let us extend the system /C to a 
full set of local coordinates in a neighbourhood of Xo by a collection of arbitrary 
smooth functions w(x) = (wi(x),..., W(oim M-3i )( x )) e R dimA4 " 31 . We only 
demand that the differential dw at ;co has maximal rank and that w(x o) = 0. 
On the space jjd™^ 4 - 31 we define the standard Euclidean norm. 

In the coordinates (/C, w ) the Hamiltonian system of ODEs with Hamiltonian 
H can be written as 

j|/C m = {GX m } + {F r ,K. m }u r , rn < 4; 

< = a(/C, w) + /3 r (AC, w)u r ; (29) 

F r u r —► max. 

Here a and /3 r are some smooth functions. Note also that the brackets /C 5 of fifth 
order are smooth functions of the local coordinates (/C, w) in a neighbourhood 
of Xo- 

The blow-up of the singularity will be performed similar to the blow-up of 
the origin in the model problem as in Section 13.21 To this end we define the 
action gn(x o) of the group M \ {0} on the space R dlmM = {(A', w)}, as follows. 

Definition 11. Let A be an arbitrary non-zero number. Define the action 
9h(x o)(A) by 


g H {x 0 )(A) : (AC 1 ,/C 2 ,/C 3 ,/C 4 , w) ^ (A 4 ^ 1 , A 3 /C 2 , A 2 /C 3 , A/C 4 , Xw). 


Of course the action gii{x o) does not preserve the Hamiltonian system with 
Hamiltonian H. But the principal part of the system can be isolated by a 
blow-up of the singularity at xo by means of the action gn{xo)- To this end 
we introduce notations for the local coordinates in the neighbourhood of the 
strange point xq. Here and in the sequel we shall set F 0 = G and 


K = F r , K, 2 ir = {F u K} r }, K? jir = {F j ,Kl}, JC% xr = {F kl JC% r }, 


where the index r runs through {1,2,3}, and the other indices i,j, and k can 
additionally take the value 0. The local coordinates will be w, /C}, /C? r , /CL , and 
r 4 

kjir’ 

Taking Remark [T4l into account, we set 


g(K) = ( (—H rr '/C}/C },) 3 + (-B rr '/C 2 0 r JC 2 0 r ,y+ 

, . S. 1/24 

+(-B rr /C 3 0 r /C 3 0r ,) 6 + (~B rr ICtooMoor') 12 ) . 


(30) 


28 The dimension 31 can be obtained as follows. The dimension of a free nilpotent Lie 
algebra of order 4 with three generators G, Fi, and F 2 equals 32. However, we excluded G 
from the set of local coordinates, and the dimension is reduced by 1. 
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where the matrix B rr denotes the matrix of the form on U* which is the inverse 
of the negative definite form B rr ' on U from Definition [9] of strange points. We 
shall also set 


(IC,w) = g H {xo) (l//x(/C)) (/C,w). (31) 

The variables /C = {/C m ,m < 4} and w lie on the spherical cylinder 

C? = {(K,w): n{K) = l}. 

The cylinder C H will be constructed as in Definition [5] in Section 15721 

Denote the map (/C,u>) H > (n,1C,w) by B. Topologically this blow-up pro¬ 
cedure is equivalent to the removal of the submanifold on which the primary 
Poisson brackets vanish and gluing in the spherical cylinder Cq in place of the 
strange point xq. 

In the sequel we denote the division of a Poisson bracket by a corresponding 
power of /u by a wide tilde. For example, we have 

jC m = K m /ii b - m or {GX n } = {G 1 IC m }/^- m . 

For brackets up to 4-th order we obtain a local coordinate on C H , and the 
brackets of 5-th order do not change under the tilde operation. 

Let us now carry over the Hamiltonian system of ODEs to the space C H , as 
we have done in Section RHil By differentiating (1501) and (1311) along an arbitrary 
trajectory we obtain 


dt p l 

fF 

F r u r 


where 


i ({G^A>} + {F\JC m }u r - (5 - rn)TK m ) ; 
i(a + /3 r u r — T w); 

max; 


(32) 


24T(/x, K, w, u) = -6 (B rr ' K.lK.1,) 2 B rr ‘ K} r ({G, Ej,} + {FXAC),K") + 
+8(B^jcl r iCl,) 3 B^'iC 2 0r ({G^ r ,} + { W^ (K ") + 
+12(B-'/C3 Or /C3 Or 0 5 B"'/C3 Or ({Gg^,} + {wf 0r 'K") + 

+24:(B rr 'K.Q 00r ICQ 00r , ) n B rr 'JC q 00j . ({GJC^ 0r ,} + {F r ^jq o0r ,}u r "). 

(33) 

As in Section [fTTH let us denote the vector field on C H D {/u > 0} defined by 
the right-hand side of (15^1) by £h(xq)- Formally the field £h(x o) is defined in a 
neighbourhood of Cq for /i > 0 only, but we extend it to the lower half of the 
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neighbourhood, i.e., to C H D {/r < 0}, by the same formula. More percisely, the 
map 


B 1 : (/x, /C, w) K > (/C, w), where (IC,w) = < 7 ff(a;o)(/x)(/C, w), 

is a 2-fold covering of a neighbourhood of Cq in C H fl {fx ^ 0} over its preimage 
in the pointed neighbourhood^! of Xq. It allows us to carry over the values of 
the functions a and /? and the Poisson brackets of 5-th order on the lower half 
C H fl {/i < 0} of the cylinder. In this case the map B 1 takes the vector field 
of system © to the field of the Hamiltonian H for /r > 0 as well as for fi < 0. 

Note that the field £,h{x o) grows like ^ as /j, —> 0. However, the field h{xo ) 
can already be continuously extended on the section Cq = C H fl {fx = 0} of the 
cylinder at all points except those in the set S H = {K* = K^,r ^ r'}. We 
shall hence assume in the sequel that the field /x£h{x o) is defined on Cq \ S H 
by means of the natural extension by continuity. Surely we have Ix£h{xo) 0 
on C^, regardless of the fact that fx = 0 on C^. The trajectories of the field 
h£h(x 0 ) either do not intersect the section Cq or lie on this section, because 
along the trajectories of the field ^h{x o) we have jx = fxT. Moreover, on Cq 
those components of the field I-i£,h(xq) which correspond to the variables (/C, w) 
are independent of fx. 

Therefore the integral curves of the fields £h(xq) and ^h{x o) coincide on 
C H C\{fx ^ 0}, but the velocity of movement along them is different. If we denote 
by s the time parameter along the trajectories of the field [x£,h{ xo), then s and 
t are related by 


System 


can then 

d „ 


ds 

d 


-fw 

as 


F r u r 


ds = —dt. 

be rewritten in the form 

= /iT (ix,jC,w,u)-, 

= {GJC^} + {FVJC m }u r 
= a + fd r u r — Tf v; 

—> max. 
ue o 


(5 — m)T/C m ; 


(34) 


6.5 Model optimal control problem on the zero section Cq 

Consider an arbitrary trajectory x(t) which enters the strange point Xq = x(0) 
as t — > —0. In this case we have /x(x(t)) < c'\t\ — > 0 as t —> — 0 by virtue of 
Lemma l23l Therefore if x(t) lies in the set B ~ l (C H \ Cq) for small t < 0, then 
the image of the trajectory B(x(t)) on C H D {/u > 0} will tend to Cq as t — ¥ —0. 

Of interest for us are those trajectories x(t) entering xq = x(0), for which 
at least one of the primary Poisson brackets /C m (x(t)) has the largest possible 

29 Note that the image of a neighbourhood of Cq in C H P {// t 9 0 [ under the mapping B 1 
does not contain the pointed neighbourhood of the point xq. 


88 





order \t\ 5 ~ m as t — > —0. In this case /j,(x(t)) has order |t| as t —» —0. Namely, 
by virtue of Lemma [251 we have c"\t\ < fi(x(t)) < c'\t\ for small t < 0. Therefore 
the trajectory x(t) does not leave the set B -1 (C' H ' \Cq) for small t < 0, and its 
image B(a;(t)) on C H D {/r > 0} tends to Cq . In fact, a stronger assertion holds. 
For t — > — 0 the image B (x(t)) tends to the submanifold Vq C Cq on which all 
secondary Poisson brackets vanish, 

Vq = |/x = 0 and /C m = 0, where /C m runs through all 
secondary Poisson brackets \ C Cq . 


Moreover, every trajectory of the field /r£#( Xq) that commences in Vq will 
never leave this submanifold. Indeed, on Vq we have on trajectories of the 
vector held ^h{xq) that = 0 and J^/C m = 0 , where /C m stands for an 
arbitrary secondary Poisson bracket in the descending system. 

System (1341) of ODEs has the same form on Vq as in the blown-up system 
m of the PMP in model problem ©■ Indeed, let us rename the primary 
brackets of the descending system by 


Vv=/Ci, x r = B rr 'jC 3 00r „ r = B rr 'lC 4 000r ,, (35) 

where (f>,il> £ U* and x, y £ U. Then we obtain the following system of ODEs 
on Vq 


it r ~ 

= 

fx r = 
as 

V = 

-fw = 

ds 

(f) r 

- 4T^v; 

—B rr 'X r 

1 

co 

H 

4* 

yT 

- 2Tx r ; 

u r 

- w, 

a(x 0 ) + Pr(xo)u r 

- Tw; 

ll) r U r — > 

max. 

u(z£l 



If we set the secondary brackets in (1551) to zero, on Vq we obtain 


(36) 


24T| d h = - 6(-£ rr V r ^) 2 (-£ rr Vr<(v) + 8 (~B rr ' ( j ) r 0 r ,) 3 (cj> r x r ) + 

+ \2(-B rr ,x r x r 'f (~B rrl x r y r ') + 2A(-B rr ,y r y r ') 11 (~B rr ,Yu r '). 


We see that if we neglect the variable w, then the system of ODEs on Vq 
which is generated by the original Hamiltonian system with Hamiltonian H 
coincides after blow-up with Hamiltonian system m of the Pontryagin max¬ 
imum principle for model problem ©, given we define the scalar product on 
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U by means of the form —B rr i. The system of ODEs which we obtain on Cq 
is in some sense a ” nilpotentization” of the original Hamiltonian system in the 
neighbourhood of the strange point Xq. 

Let us remark also that the property of the triangle SI of being equilateral is 
determined by the form —B rr i. Namely, the triangle Q is equilateral if the form 
B rr f is as described in Theorem 0 Indeed, the vertices of the triangle SI given 
by (1271) have to form an equilateral triangle with respect to the form — B rr > 
with center in the origin. Hence the Gram matrix of the vertices of S 1 has to be 
proportional to matrix (1251) with a negative proportionality coefficient, whose 
modulus equals the side-length of the triangle S2. 


6.6 Nilpotentization in the neighbourhood of the strange 
point 


Below we shall show that the behaviour of the original Hamiltonian system is 
described by its behaviour on the zero section of the cylinder C. The proof 
is based on a thorough consideration of the Poincare map in system (1341) in the 
neighbourhood of points of the strange sets S and X from Theorems [5] and 0 
As an auxiliary system of ODEs in the neighbourhood of xq we consider the 
system obtained from system (1M1) by ’’nilpotentization” with respect to ji by 
means of the blow-up map B. 

The nilpotentization procedure in the neighbourhood of the strange point by 
means of the blow-up map is performed as follows. We transform the right-hand 
side of system (l34l) . which was obtained by the blow-up map B, as follows. The 
differential equations on the variables /C and w are extended without modifica¬ 
tion from the zero section Cq ot the whole cylinder C. The right-hand side of the 
differential equation on /i is linearized with respect to /i in the neighbourhood 
of the zero section. This means terms of order o(/u) are discarded. As a result 
we obtain the system 


m 

d, tr2 

Ar^ 

ds_^ r 

2L/r 4 

ds'z OOOr 

A AT 4 

ds kjir 

Aw 

ds 


/iT(0, 1C, w, it); 

Kl r + K? r , r u r ' - 4£jT, 

JCl ir + n, ir A -3JCI T, 

IC^+lC^y - 2TC% r T, 

B rr 'U r — K4 0r T, 

kjir Tj if fctjir - secondary bracket 
a(a"o) + (3 r ( x o) ur ~ Y(0, 1C, w, u)w; 

max, 

u£Q, 


(37) 


where here and below the summation over the upper and lower index r' runs 
over r' = 1,2,3. Let us rewrite this system in the original coordinates /C and 
w, which have been introduced before blow-up. Applying the inverse map B _1 
and transforming back the time parameters s and t we obtain 
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^T^-GOOr — B rr tU r , 

jjjK'kjir = 0, if fcijir is a secondary bracket 


jfew = a(x 0 ) 


w = a(xo) + /3 r (x 0 )u r ; 


( 38 ) 


K,].u r —> max. 




Hence the ” nilpotentization” by means of the blow-up map B with respect to 
H in the neighbourhood of the strange point leads to system (l38l) . This system 
can be obtained from system (l29l) by replacing the functions a and p r and the 
brackets of 5-th order by their values at the point xq. 

As mentioned before, system (1371) coincides on the zero section Cq with 
system The differential equations on /C and w are independent of /r, and 

the right-hand side of the equation on /r is obtained by linearization of (1321) with 
respect to n in the neighbourhood of Cq = C H D {y = 0}. 

Let us note the following important facts about system (EH): 

1. The right-hand side of the differential equations on K, is independent of w 
and y. After the coordinates /C on the trajectory of system (1371) have been 
found, the variables In p and w can be calculated by direct integration. 

2. If the control is constant, then the evolution of the secondary Poisson 
brackets is uncoupled from the evolution of the primary Poisson brackets. 
The primary Poisson brackets influence only the time instant of the control 
switching. 

Hence every trajectory of the original Hamiltonian system with Hamiltonian 
H which lies in T>q can be found also in the system determined by model 
problem ©■ Indeed, consider an arbitrary trajectory (x(t), y(t ), 4>{t), ijj(t),u(t)) 
of the PMP of model problem m lying in M + (see Corollary [TJ), i.e., entering 
the origin. By virtue of Theorem [l] this trajectory enters the origin at a time 
instant t = T(x(0),y(0)). By virtue of (l35l) this trajectory determines the 
primary Poisson brackets /C as functions of time t for t <T (x(0), j/(0)). In order 
to obtain a trajectory of system (1551) we still need to determine the secondary 
Poisson brackets /C and the variables w as functions of time. To this end we 
set the secondary Poisson brackets identically zero and determine the function 
w(t) from the last equation of system (l38l) . 



Since we are interested in trajectories which enter the strange point xo, we 
also set wo = w(0) = 0. The constructed trajectory satisfies the ODEs of 
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the ’’nilpotentized” system (1551) . Hence we have constructed a map n + from 
M + C {(x,y, cf>,i/>)} into the space {(/C,u>)} by means of the formulas 



Lemma 24. The map n + : M + —» {(/C, w)} constructed above is an equivariant 
injection, i.e., it is compatible with the action of the group R+, 

n + ° gW = 9h(x 0 )(A) o n+. 

Moreover, the map n+ is continuous on M+ and takes the trajectories of Hamil¬ 
tonian system m of model problem m on M_|_ to trajectories of the ’’nilpoten¬ 
tized” system \38\) . 

If in addition dG(x o) = 0, then the map n + is locally Lipschitz. 

Proof. By construction the map n + is equivariant and injective. Continuity of 
n+ on M .|_ \ {0} follows from Theorem [T) In this theorem it is proven that 
the function T(-) is continuous. Also by construction the map n + takes the 
trajectories of system m to the trajectories of system (1551) . 

If dG(x o) = 0, then x{t) = Xq is a trajectory of the original Hamiltonian 
H = G + F r u r for u = 0. Hence JC(t) = 0, w(t) = 0 is a trajectory of system 
(ESI for u = 0. Then the condition a(xo) = 0 is necessary and the map n_)_ 
does not contain a summand with T(a:(0), y( 0)). It is hence locally Lipschitz by 
virtue of Theorem [1] □ 

Thus we can find every trajectory of system (j3j in the ’’nilpotentized” system 
(1551) . However, in the original system (1551) these trajectories may not exist. 
However, it turns out that the trajectories from the sets X (Theorem [6| and S 
(Theorem [2| can still be found in the original Hamiltonian system. To this end 
we have first to project the sets n + (A) and n + (S) on Cq , and then to lift each 
trajectory from the zero section Cq by means of the Hadamard-Perron theorem. 

By the equivariance of the map n + the map TL + /g : M + /g —> Cq is well- 
defined. Moreover, the image of M + /g lies in Vq. Since the vector field /j,£h(x o) 
of the blown-up ” nilpotentized” system (1571) and the vector field of the blown- 
up model system (1151) are also preserved by the action of the group R\ {0}, we 
have that every trajectory of the field gf on M + /g is taken to some trajectory 
of the field gfii(x o). 

Let us now transfer the sets X and E from the model optimal control problem 
to the zero section Cq . Define 


X° h (xq) = (n + /g)(X/g) C Vq ; and ^° H (x 0 ) = (H + /g)(E/g) C Vq . 
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Note that the set Xh(x o) is defined for the Hamiltonian system with Hamil¬ 
tonian H only if the triangle f 1 is equilateral with respect to the form B rr f. In 
other words, it is defined only if the form B rr r has the form described in the 
conditions of Theorem [T| 

6.7 Poincare map in the Hamiltonian system 

In the previous section we have constructed sets X^ i {xq) and E^ixo) on the 
submanifold T>q C Cq for a generic Hamiltonian system with Hamiltonian 
H. However, the inverse map B 1 will take the section Cq entirely to the 
strange point Xg. In order to find the corresponding trajectories in the original 
Hamiltonian system, we have to apply the Hadamard-Perron theorem to each 
(hyperbolic) trajectory of the sets Eh(x o) and Xh (xq) on T>q . We will show that 
their stable submanifolds do not anymore lie in the zero section Cq . The union 
of these stable submanifolds will yield the sought sets Xh(xq) and Eh{x o). The 
unstable submanifolds lie in Cq and have no analog in the original Hamiltonian 
system. 

We shall apply the Hadamard-Perron theorem to the Poincare map in the 
neighbourhood of an arbitrary trajectory 7 from X^^xo) or E° H (xo )• Every 
trajectory (x(t) ,y(t) , <p(t) , from the set S or X in model problem © 

intersects the switching surface only transversally. Hence in the neighbourhood 
of any of its points a smooth Poincare map is defined. In this section we transfer 
the structure of this map from the model problem to the trajectory 7, first in 
the ”nilpotentized” system (l37l) . and then also in the original system (l3dl) . 

We are interested in the differential of the Poincare map &h from the switch¬ 
ing surface to itself in the ” nilpotentized” system (l37l) in the neighbourhood of 
an arbitrary trajectory in T>q . The Poincare map in the original Hamiltonian 
system will be denoted by $//, without the overbar. 

In order to construct the map in the neighbourhood of the switching 
point zq = 7(0) on the trajectory y(t) from X^^Xq) or E 0 h (xq), we use the follow¬ 
ing simplifying consideration. Since the trajectory 7 (t) intersects the switching 
surface S H transversally consecutively at the points zq and zi = §h(zo) = 7(7"), 
the control is also constant between the intersection points with S H for any tra¬ 
jectory in the neighbourhood of 7(f), and coincides with the control on 7 (t). 
Hence we may assume that the control in the ’’nilpotentized” system eh is 
constant and compute d$>H as a composition of the linearization of the evolu¬ 
tion of system (13711 for the fixed time interval r with the projection onto the 
tangent plane to S H along the vector field of system (Ell). Let us denote the 
evolution map by &h(t) and compute it. 

The equation of variations for system E3 in the neighbourhood of 7 is of 
the form 


H{t ) = 

Since the trajectory 7 lies in Vq , not all primary brackets vanish on 7 . Since 
the function T is independent of /i and the secondary Poisson brackets K , the 
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matrix M must have the following block structure. (Here and in the sequel we 
shall write T (t) = Y(y (t)) for brevity. The identity matrix of any appropriate 
size is denoted by I.) 


/./Primary brackets /C Secondary brackets 1C 


m(t( t) ' 

0 

0 

0 

0 

0 

0 

0 

0 

Primary brackets 

0 

* 

* 

* 

* 

* 

* 

* 

0 

Primary brackets 

0 

* 

* 

* 

* 

* 

* 

* 

0 

Primary brackets 

0 

* 

* 

* 

* 

* 

* 

* 

0 

Primary brackets ^000r^ 

0 

* 

* 

* 

* 

* 

* 

* 

0 

Secondary brackets /C^ ^ 

0 

0 

0 

0 

0 

-3T (t)I 

* 

* 

0 

Secondary brackets 

0 

0 

0 

0 

0 

0 

-2T (t)I 

* 

0 

Secondary brackets /C^ ^ 

0 

0 

0 

0 

0 

0 

0 

-T (t)I 

0 

w{ 

0 

* 

* 

* 

* 

* 

* 

* 

—T(t)J 


The matrix M is block-lower triangular with three blocksjm the diagonal, 
here the central block, which corresponds to the variables /C, is block-upper 
triangular. Therefore the matrix <f>#(t) possesses the same structure, 

/-^ Primary brackets K. Secondary brackets K. 'HJ 

a*{T t t t 


Primary brackets 


AC{ 0 


* 


* 


0 


Secondary brackets 


Ac{ 0 


0 c 


0 



* 


* 


w 


Here the matrix W is proportional to the identity matrix, and the matrix C is 
upper triangular. 

Let us explicitly write out the differential equations on A, W, and C : 


j f \ = Y(f)A, A(0) = 1. 

Denote A(r) = p , where r is the time on the trajectory 7 from z 0 to z\. Further, 
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^-w = -T(t)W, W( 0) = I =► W(t) = p" 1 /. 

The matrix C obeys the equation 

/ -3T (t) * * \ 

T C = 0 -2T (t) * C, C(0) = I. 

a ' \ 0 0 -T (t) ) 

Hence the diagonal blocks of C are given by p~ 3 I , p~ 2 I, and p -1 /. 

Let us now compute the number p. Let the trajectory y 7 of system (1371) be 
a lifting of 7 from Cq = {p = 0} to {p / 0}. In other words, the coordinates /C 
and w coincide on 7 and 7 ', but on 7 ' we have p 7 ^ 0. Denote by z r 0 and z[ the 
corresponding liftings of zq and Z\. 

Lemma 25. Denote po = p(z' 0 ) and pi = p(z[). Then we have 

hi 

— = P- 

ho 

Proof. Indeed, pi is a function of the initial point z' 0 = (po,/Co;^o), be., pi = 
pi (po, /Co, wo). Clearly we have 

pi(Kpo,Xio, w 0 ) = Kpi(p 0 ,^o, w 0 ) Vk G R. 

Differentiating this relation with respect to n at k = 1 we obtain 

dpi _ pi 
dpo Po ’ 

The right-hand side of this equation does not change if po is multiplied by 
k/ 0. We hence get that the derivative is constant on the whole vertical 

line (kpo,/C o) wo), s£l. However, on 7 , i.e., for k = 0, we have that = 
A(r) = p. This completes the proof. □ 

Remark 15. If we discard the coordinates w and the secondary Poisson brackets 
on the trajectory y 7 , or equivalently, if we apply an inverse of the map n + /p, 
then we recover an optimal trajectory of model problem ©■ Therefore we 
can apply the key Lemma [12] to it, which guarantees an exponentially fast 
convergence to zero of the coordinate p(s) on this trajectory as s —> + 00 . 

Let us now consider the projection operator of the tangent space T Zl C H onto 
the hyperplane T Zl S H along the vector field of system (El). The vector field of 
system (l37l) is discontinuous at z 1 , and we are interested in the velocity vector 
of the trajectory 7 when it arrives at z\ for t —> t — 0. Denote this vector by v. 
Then we have 
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The co-vector a defining the hyperplane T Zl S H has the form 

a = ^ 0 * | 0 | 0 

Hence the projection operator 1 — v ^ ® onto T Zl S H along v has the form 


P 

p(i 


( v) 


Primary brackets K. Secondary brackets AC 


Primary brackets AC | 0 


Secondary brackets AC{ 0 


w 


0 


0 


Thus the differential of the Poincare map of the ” nilpotentized” system 
(EH at the point Zq is given by 


d$H\z 0 = (1 — 


( v®a\ 

l 1 -rr 

V a(v) / 


and has the block structure 


1 p 

0 

0 

0 ^ 

0 

A 

* 

0 



P~ 3 I * * 


0 

0 

0 p~ 2 I * 

0 



0 

0 

1 


V 0 

* 

* 

p-H j 


Remark 16. Note that the matrix A is obtained by conjugation with the differ¬ 
ential of the map (1351) from the corresponding matrix in the PMP system EJ for 
model problem ©■ This follows immediately from Section 15751 Moreover, since 
all considered trajectories lie in T>q the matrix appearing in the conjugation 
will be constant and independent of the choice of the trajectory 7 on T>q . 

Let us now describe the matrix of the differential of the Poincare map for the 
original, not ’’nilpotentized” system (l34l) . Since the original system (l34l) and the 
’’nilpotentized” system (1371) coincide onC^, the matrices d$n and d$H have the 
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same elements everywhere except the first row and the first column. However, 
the first column is also identical, since the differential equation on /i differs only 
by terms of order 0(/i 2 ) in the systems (1511) and (1571) in the neighbourhood of 
C H 

Hence we have proven 


Lemma 26. The differential d&H of the Poincare map for system i‘H\) at the 
point zq € T>q fl S H from Xfj(xo) or S^(xq) on the surface S H has the form 


p Primary brackets K. Secondary brackets /C 


W p 0 0 0 0 


0 


Primary brackets /C j. ^ '■¥ 
Primary brackets /Cg r ^ 'if 
Primary brackets ^OOrj * 
Primary brackets * 

Secondary brackets K . v ^ 


A 


Secondary brackets * 0 0 0 0 


w\ * 


0 


0 


0 0 0 0 p~ 3 I * 

Secondary brackets K? * 0 0 0 0 0 p~ 2 I 


0 


0 


p- 1 / 


w 

T 

0 

0 

0 

0 

0 

0 

0 

p- 1 / 


where the matrix A is up to substitution \35\) equal to the corresponding matrix 
for system m of the Pontryagin maximum principle for model problem m, and 
p is obtained by virtue of lemmaUISl 


6.8 finalization of the proof of the chaotic behaviour of 
the trajectories in a generic Hamiltonian system 

Lemma [55] allows us to use the results on the chaotic behaviour of the optimal 
synthesis for model problem JT]). As we mentioned above, the sets S^(xo) C Cq 
or Xfj(x o) C Cq are constructed on the zero section Cff and are taken to the 
strange point xo by the inverse map B -1 . In order to preserve the chaotic 
behaviour of the synthesis, we shall for every optimal trajectory from X or 
S construct two-dimensional stable submanifolds by virtue of the Hadamard- 
Perron theorem @]. These will leave the zero section Cq . More precisely, we 
shall apply the Hadamard-Perron theorem to the Poincare map along the 
trajectory. 

Let us fix a trajectory 7 from the set E 0 h (xq) or the set X^(xq)- The 
trajectory 7 is the image of some trajectory 7 from 5/p or X/g under the map 


97 




II +/g. Let us denote the intersection points of the switching surface S H with 
the trajectory 7 consecutively by ..., Z-i, Zo, Zi ,..and with the trajectory 7 
by ..., Z—i, zqj'zi, ■ ■ ■■ The matrix A and the number p from Lemma 1261 which 
correspond to the fc-th switching point Zk will be denoted by A(zk) = Ak and 
p{zk) = Pk, respectively. By virtue of Lemma 1261 we have for Hamiltonian 
system © 

= ( o‘ l ) ■ 

Here and in the sequel $ denotes the Poincare map in the blown-up model 
system with vector field where £ is the vector field of system USD- 

First we show that the original Hamiltonian system with Hamiltonian H is 
contracting in the vertical direction. The differential d<$>H{zk) corresponding to 
a single application of the Poincare map <f># may i n general have an eigenvalue 
Pk which is larger than 1. We claim, however, that Lemma 1121 guarantees the 
existence of a natural number N such that is contracting in the vertical 
direction. Indeed, every optimal trajectory of the model problem enters the 
origin in finite time. Thus we will eventually obtain a contraction in the vertical 
direction if we repeat the Poincare map <&h often enough. We shall now prove 
this claim formally. Let us use the results obtained for model problem ©■ 
From Lemma [1^1 it follows that on the optimal trajectory 7 on M + we have the 
inequalities 


Cie 


-C2S < 


mOvPO) 

Mt(o)) 


< c 3 e 


where the positive constants Cj , k = 1,..., 4 are independent of the choice of 
trajectory on M + . Therefore 


u(s) 1 1 , . 

—— < - when s > — ln(2c 3 ). 
p(0) 2 c 4 

We now bound the transition time <7(2) between two consecutive intersections of 
the switching surface S at points z € S and $( 2 ) £ S when measured with the 
velocity of the field /r£ along an arbitrary trajectory on S or X . The function 
cr is continuous and independent of p, and the sets (S/g) fl S and (X/g) fl S 
are compact and have an empty intersection with <Si 2 3 . Therefore we get the 
bounds 


Smin ^ ^(-^) — '-’max 

for some nonnegative constants s m i n and s m ax- Since any continuous function 
achieves its minimum on a compact set, we have s m ax > Smin > 0. Hence we 
get by virtue of Lemma [25] 

PkPk+i ■ ■ ■ Pk+N-i < \ Vfc for N > — -— ln(2c 3 ). 

5minC4 
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Thus, if we fix a sufficiently large number N and work only with the iV-th 
power of the maps $ and instead of these maps themselves, we may assume 
that we have a contraction in the vertical direction. More precisely, we have 
proven 

Lemma 27. Let us ffJ^I N > —-— ln( 2 c 3 ) and denote 0 = and Oh = 

s min c 4 v ' 

$h N - Set also g(z kN ) = Qk = ( PkN-iPkN -2 ■ ■ ■ P( k -\)N)~ l > 2 and A{z kN ) = 
A k = (^fcAr-i^ 4 fcJV -2 • • ■ ^(fe-i)^)” 1 - Then we have 

dO{z kN ) = ( g fc Ak ^j, 

and the matrix dOn{z k N) has the structure described in Lemma\2 6] when re¬ 
placing p by g k and A by A k . 

As a next step towards the application of the Hadamard-Perron theorem we 
will conduct a thorough study of the structure of the matrices A k . We shall 
show that in some special decomposition of the tangent space all matrices A k 
become block-diagonal. For convenience we shall denote 

= m 

where the linear operator A(S) acts from Tb(Co (~l S ) to T$(?)Co D S. 

Lemma 28. At the points z of each of the sets (E/g) D S and {A/g) n S there 
exists a decomposition of the tangent svac^l T^(Cq fl«S) = U(z) 0 V(z), where 
dim [/(S') = 1 and dirnP(S) = 5, and a metric on the factors such that the 
operator A{z) takes the form 


A{z) 


S(z) 0 \ 

0 S(z) ) ’ 


where a(z) : U(z) —> [/($(S)) and S(z) : V(z) —> P($(S)). Here a is a con¬ 
tracting operator, and 6 is an expanding operator, i.e., 


||a(S)|| < 1 and ||<5(S) 1 || < 1 VS. 

Moreover, both the metric and the decomposition T^{Cq D S) = U(z) 0 V(S) 
depend continuously on S. 

Proof. The proof of the lemma is divided into two parts. First we choose an 
’’almost” correct decomposition of the tangent space at each of the points in 
the sets (S/g) fl S and {X/g) fl S, such that the matrix A( S) becomes ’’almost” 
block-diagonal, and then we transform this decomposition. 

30 The number N is independent of the point z k and can be chosen universally for all 
trajectories. 

31 Recall that dim Co PlS = 6. 
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We claim that in the neighbourhood of each of the sets ( E/g)C\S and (X/ g)C\ 
S there exist local coordinates and a Riemannian metric such that 


A~\z) 


a{z) P{z) \ 
7(+) S(z) ) 


(40) 


for every point z in these neighbourhoods, where o/z) is a number, (3{z) and 
7 (z) are 1x5 and 5x1 matrices, respectively, and is a 5 x 5 matrix. 
Moreover, there exist positive numbers u x , u y , v x , and v y such that 


IK+) 1 < Ux 1 , \\P(z)\\ < u y , 117(2)11 < v x , ||5(z)|| < v y . 

Here the diagonal elements are of hyperbolic nature and the off-diagonal ele¬ 
ments are small compared to the diagonal elements. More precisely, the numbers 
u x , it y , v x , and v y satisfy the inequalities 

f u x > 1 > v y ; / jj j 

\ {U x - 1)(1 - Vy) > U y V x . y 

Indeed, the existence of such local coordinates in the neighbourhood of the 
set (S/g) n<S follows from the Grobman-Hartman theorem [3] and the transver- 
sality of the homoclinic point (see Lemma 1X111) . The set (S/ 5 ) n<S from [2] can be 
reduced such that in an arbitrary metric the norm of the off-diagonal elements 
will be arbitrarily small, and the diagonal elements will be contracting and ex¬ 
panding operators of the same dimensions as for the 6 -link chain, namely 5 and 
1 . 

For the set (^/< 7 ) D <S we prove this by explicitly describing appropriate 
local coordinates on the switching surface given by the relation tpi = tps with 
normalization </>2 = 1. First we reduce the number of variables to six, namely 

TTl = - 2 / 3 ), 7T2 = ^( 2 / 1 - 252 + 53 ), 7T3 = ^{x-L-Xz), 7T4 = ^(^1-2x2 + 

X 3 ), 7 t 5 = -^(<(>1 — <(> 3 ), 7 r 6 = ^(V’l — 2-i / 2 + ^ 3 ). Now in each of the 16 domains 
AA.AA, AA.AC ,..., CC.CC we apply a linear coordinate transformation 77 = 
Ain, where the constant coefficient matrices for the 16 domains are given by 


Maa.aa 


/—0.5913 

-1.0677 

0.0848 

-1.3179 

-0.0023 

0.6606 

-0.2024 

0.8600 

0.5347 

-0.1061 

V— 1.1822 

0.5279 


0.0683 0.2937 

-0.9326 -0.6262 
0.2677 0.7432 

-0.7595 0.9189 

1.3964 0.6468 

-2.5279 -0.2730 


-0.0114 

0.0316 \ 

-0.0104 

0.0737 

0.4258 

0.2468 

-1.5610 

0.7982 

2.3193 

-2.1665 

-3.8024 

2.6844 ) 
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Maa.ac = 


Maa.ca = 


Maa.cc = 


Mac. a a = 


Mac. ac = 


Mac.ca = 


Mac.cc = 


Mca.aa = 


/— 0.3772 

- 0.6812 

0.0436 

0.0603 

- 0.9371 

- 0.6631 

- 0.0016 

0.4697 

0.1903 

- 0.1439 

0.6115 

- 0.5401 

0.3801 

- 0.0754 

0.9930 

\— 0.8407 

0.3754 

- 1.7976 

/— 0.3971 

- 0.6704 

0.1033 

0.1116 

- 0.2444 

- 0.2931 

0.0819 

0.6562 

0.5188 

- 0.0770 

0.5547 

- 0.3887 

0.2177 

- 0.0065 

0.6332 

\— 0.1329 

0.0749 

- 0.4715 

/— 0.2941 

- 0.4965 

0.0765 

0.1180 

- 0.2584 

- 0.3099 

0.0865 

0.6937 

0.5485 

- 0.0815 

0.5864 

- 0.4109 

0.2302 

- 0.0068 

0.6694 

\— 0.1405 

0.0791 

- 0.4985 

/ — 1.3472 

- 1.2301 

0.8883 

0.1653 

- 1.7355 

- 1.7124 

0.3610 

0.1926 

2.3424 

- 0.2216 

1.5281 

- 0.9719 

1.7063 

- 1.3513 

3.2009 

\— 0.4124 

0.3207 

- 1.2318 

/— 1.5370 

- 1.4034 

1.0134 

0.1456 

- 1.5288 

- 1.5084 

0.3180 

0.1697 

2.0633 

- 0.1951 

1.3460 

- 0.8561 

1.5029 

- 1.1902 

2.8195 

y — 0.3632 

0.2825 

- 1.0851 

/ — 1.4918 

- 1.2360 

1.0849 

- 0.5929 

0.5930 

1.2489 

0.2087 

- 2.0370 

0.9934 

0.8390 

1.3864 

4.1958 

1.8997 

- 0.1209 

6.1266 

\— 0.5410 

0.3880 

- 1.7381 

/- 1.4492 

- 1.2008 

1.0539 

- 0.6064 

0.6066 

1.2774 

0.2134 

- 2.0836 

1.0162 

0.8583 

1.4182 

4.2918 

1.9431 

- 0.1237 

6.2668 

\— 0.5534 

0.3969 

- 1.7778 

/— 0.8312 

- 1.5009 

0.0960 

0.1017 

- 1.5797 

- 1.1178 

- 0.0028 

0.7918 

0.3209 

- 0.2426 

1.0308 

- q .§ J 04 

0.6409 

- 0.1272 

1.6739 

\ — 1.4171 

0.6327 

- 3.0301 


0.1874 

- 0.0072 

0.0201 \ 

- 0.4452 

- 0.0074 

0.0524 

0.5285 

0.3028 

0.1755 

0.6534 

- 1.1101 

0.5675 

0.4599 

1.6493 

- 1.5406 

- 0.1940 

- 2.7037 

1.9088 ) 

0.2038 

0.0053 

- 0 . 0444 \ 

- 0.2188 

0.0594 

0.0028 

0.8690 

1.3697 

0.2361 

0.6313 

- 0.8937 

0.5253 

- 0.1496 

1.2464 

0.3705 

- 0.2392 

- 0.9990 

0.9895 ) 

0.1509 

0.0039 

- 0 . 0329 \ 

- 0.2314 

0.0629 

0.0030 

0.9186 

1.4479 

0.2496 

0.6674 

- 0.9447 

0.5552 

- 0.1581 

1.3176 

0.3916 

- 0.2529 

- 1.0561 

1.0461 ) 

0.9348 

- 0.4362 

- 0 . 0239 \ 

- 0.5395 

0.6959 

0.2788 

- 0.1152 

4.4150 

- 0.4106 

1.9245 

- 2.4828 

1.4865 

- 1.9034 

4.1924 

- 0.2951 

- 0.1906 

- 2.4337 

2.6356 ) 

1.0666 

- 0.4977 

- 0 . 0273 \ 

- 0.4752 

0.6131 

0.2456 

- 0.1015 

3.8890 

- 0.3616 

1.6952 

- 2.1869 

1.3094 

- 1.6767 

3.6929 

- 0.2599 

- 0.1678 

- 2.1437 

2.3215 ) 

0.9744 

- 0.6000 

— 0 . 1150 \ 

0.5188 

- 0.7829 

- 0.2039 

- 2.4691 

2.5841 

- 1.8853 

1.8630 

10.4128 

0.9577 

- 1.0920 

12.6540 

3.2477 

- 0.6350 

- 3.6204 

3.9319 ) 

0.9467 

- 0.5829 

- 0 . 1117 \ 

0.5307 

- 0.8008 

- 0.2085 

- 2.5256 

2.6432 

- 1.9285 

1.9056 

10.6511 

0.9796 

- 1.1170 

12.9436 

3.3220 

- 0.6495 

- 3.7032 

4.0219 ) 

0.4129 

- 0.0159 

0.0445 \ 

- 0.7505 

- 0.0125 

0.0883 

0.8908 

0.5104 

0.2957 

1.1014 

- 1.8712 

0.9568 

0.7752 

2.7801 

- 2.5969 

- 0.3272 

- 4.5577 

3.2177 j 



Mca.ac 


Mca.ca 


Mca.cc 


Mcc.aa 


Mcc.ac 


Mcc.ca 


Mcc.cc 


/— 0.5590 

- 1.0094 

0.0646 

0.0284 

- 1.2522 

- 1.0645 

0.0136 

0.5899 

0.2568 

- 0.1603 

0.7743 

- 0.6799 

0.4698 

0.0295 

1.3318 

1.1106 

0.4685 

- 2.3717 

/— 0.6987 

- 1.1795 

0.1817 

0.2866 

- 0.6276 

- 0.7527 

0.2102 

1.6846 

1.3320 

- 0.1980 

1.4241 

- 0.9979 

0.5591 

- 0.0166 

1.6256 

\— 0.3412 

0.1922 

- 1.2105 

/— 0.5165 

- 0.8721 

0.1343 

0.2310 

- 0.5057 

- 0.6066 

0.1694 

1.3576 

1.0734 

- 0.1595 

1.1476 

- 0.8042 

0.4506 

- 0.0134 

1.3100 

\— 0.2750 

0.1549 

- 0.9756 

/— 1.7029 

- 1.5549 

1.1228 

0.2242 

- 2.3541 

- 2.3229 

0.4897 

0.2613 

3.1773 

- 0.3006 

2.0727 

- 1.3182 

2.3143 

- 1.8329 

4.3417 

\— 0.5593 

0.4349 

- 1.6709 

/— 2.1389 

- 1.9529 

1.4103 

0.2145 

- 2.2524 

- 2.2225 

0.4685 

0.2499 

3.0400 

- 0.2876 

1.9832 

- 1.2612 

2.2144 

- 1.7536 

4.1542 

\— 0.5352 

0.4161 

- 1.5988 

/— 2.1848 

- 1.8102 

1.5888 

- 0.9631 

0.3038 

0.8826 

0.1995 

- 2.1397 

1.4883 

0.9908 

1.4063 

4.5037 

2.2134 

- 0.2631 

6.8269 

\— 0.6009 

0.2926 

- 2.2588 

/— 1.8994 

- 1.5738 

1.3813 

- 0.6747 

0.6749 

1.4212 

0.2374 

- 2.3179 

1.1304 

0.9548 

1.5775 

4.7743 

2.1615 

- 0.1375 

6.9713 

\— 0.6155 

0.4415 

- 1.9777 


0.2777 

- 0.0107 

0.0300 \ 

- 0.6260 

- 0.2454 

0.2924 

0.6980 

0.4571 

0.2243 

0.8766 

- 1.3837 

0.6740 

0.6845 

2.0995 

- 2.0201 

- 0.2692 

- 3.5420 

2.4780 j 

0.3586 

0.0093 

- 0 . 0782 \ 

- 0.5620 

0.1527 

0.0072 

2.2311 

3.5163 

0.6063 

1.6208 

- 2.2944 

1.3485 

- 0.3841 

3.1999 

0.9511 

- 0.6143 

- 2.5648 

2.5404 j 

0.2651 

0.0068 

- 0 . 0579 \ 

- 0.4529 

0.1230 

0.0059 

1.7980 

2.8339 

0.4887 

1.3062 

- 1.8490 

1.0867 

- 0.3096 

2.5789 

0.7664 

- 0.4950 

- 2.0670 

2.0473 J 

1.1818 

- 0.5513 

- 0 . 0303 \ 

- 0.7318 

0.9440 

0.3782 

- 0.1562 

5.9887 

- 0.5568 

2.6104 

- 3.3677 

2.0164 

- 2.5818 

5.6867 

- 0.4003 

- 0.2585 

- 3.3011 

3.5749 J 

1.4843 

- 0.6925 

— 0 . 0380 \ 

- 0.7003 

0.9032 

0.3618 

- 0.1494 

5.7300 

- 0.5328 

2.4976 

- 3.2221 

1.9292 

- 2.4703 

5.4411 

- 0.3829 

- 0.2473 

- 3.1585 

3.4205 j 

1.4272 

- 0.8787 

- 0 . 1684 \ 

0.2771 

- 2.6832 

- 1.3049 

- 2.6630 

3.2584 

- 1.9360 

1.9791 

11.6620 

0.8665 

- 1.3631 

14.3844 

3.5996 

- 0.8628 

- 4.4148 

4.3259 j 

1.2407 

- 0.7640 

- 0 . 1464 \ 

0.5903 

- 0.8909 

- 0.2319 

- 2.8096 

2.9404 

- 2.1454 

2.1199 

11.8485 

1.0898 

- 1.2427 

14.3987 

3.6954 

- 0.7225 

- 4.1196 

4.4740 / 


In these coordinates the constants u x , u y , v x , v v for the Poincare map on the 
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domains AA.AA, AA.AC, ..., CC.CC satisfy the inequalities 

\u x | > 1.2641, \\tty || < 0.3894, ||^|| < 0.3894, ||v y || < 0.4000. 

From now on the reasoning will be identical for both sets (S/g) fl5 and 
(X/g) D S. We shall then denote any of these sets by X. 

Thus for every point z in X we have a decomposition of the tangent space 


T s (C 0 nS) = U(z)®V(z), 

depending continuously on z € X. We have that a(z) : U(z ) —> C/($ _1 (?)), 
P{z) : V(z) -> U($~ 1 (^)), 7(2) ■ U(z) -> V($ _1 (2)), and 8{z) : V{z) -> 
V(<I> _1 (z)) are linear operators. In our case we have dim U(z) = 1 and 
dim V(z) = 5. This is not important, however, and our considerations below are 
valid for arbitrary dimensions dim U(z) = m\ and dim V{z) = m 2 ■ Accordingly, 
we shall consider a(z) not as a number, but as an invertible linear expanding 
operator which dilutes by a factor of at least u x . Vectors in the spaces U(z) 
and V(z) will be denoted by u and v. 

We shall now seek a linear coordinate transformation v = v + 7Z(z)u in each 
of the spaces T^(Cq D S) such that the lower left element of the matrix A -1 (2) 
vanishes. Let us compute 7 Z(z). We have 


u \ _ ( 1 0 \ / u 

v ) ~ V 1 ) V v 


hence 




or in other words 


1 0 


a(z) (3(z) 
7(2) 8{z) 


1 0 
-lZ(z) 1 


A - (2) 


«( 2 ) 0(z) 

7(2) S(z ) 


where 


( a(z) = a(z) - f3{z)lZ{z) 

I fi(z) = /3(z) 

I 7 (z) = 7 (z) — 8(z)lZ(z) + TZ^" 1 (z))a(z) — 7£(3> _ 1 (2))/3(2)7t!.(2) 
{ S(z) = 8(z) +n$- 1 (z))p(z). 


We look for a substitution such that 7 (z) = 0 for all z in X. We then get 

^($ _1 (2)) = -(7(2) - 8(z)7Z(z))(a(z) - £(2)72.(2)) _1 . (42) 

Let us look for an operator 7 Z(z) of bounded norm not depending on z, i.e., 
11^(2)11 < £ for some fixed e > 0 and every z £ X. We now precisely choose 
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£ > 0. In order for the operator a(z) — f3(z)lZ(z) to be invertible it is sufficient 
that 


u x — £Uy > 0 . 

In order for the map at the right-hand side (Hi?l) to define an operator of 
norm not exceeding e it is sufficient that 

Vx + ev y < 

U x £Uy 

Let us now demand another condition on s, namely such that the map at 
the right-hand side of equation (l42l) is contracting. The differential of this map 
is given by 


(<5(z) + 1 (z))/3(z))d'R,(a(z) - P(z)K(z)) 

Hence a sufficient condition for the map to be contracting is 


Ux £Uy 

As a last condition on e we demand that the operator A -1 (2) preserves the 
hyperbolic structure, namely such that ||5(z) _1 || < 1 and ||<5(z)|| < 1. Sufficient 
conditions for this are 


- < 1 and v v + eu v < 1 . 

U X £Uy 

Gathering all conditions on e > 0, we obtain the system 
Uy£ 2 - (u x - Vy)e + v x < 0; 

< u y e < \{u x - v y ); (43) 

u y e <u x - 1; 

Uy£ < 1 — Vy. 

Let us show that this system has a feasible solution e. If u y = 0, then the 
system has the trivial solution e > - v * v . Let us now assume u y > 0. The first 
condition of system (14311 is a consequence of the fourth, and the third one is a 
consequence of the 4-th and the 5-th one. After the substitution C = u y £ we 
then obtain the equivalent system 

{ C ( ria ; Vy'jC “(“ UyV X 'd 

C <u x - 1; 

C < 1 — Vy. 

The discriminant of the quadratic trinomial in the first inequality is positive. 
Indeed, from (14T1) we have 

D — (u x Uy) 4 UyV X ^ 4(Wa; l)(l ^y) 4 UyV X 0. 
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Therefore the system has a feasible solution if the smaller root of the quadratic 
trinomial is smaller than both u x — 1 and 1 — v y . We obtain 


'U'X Vy 

2 


Vd 


< u x - 1 


or equivalently 

VD >2 — u x ~ Vy 
It remains to show that 


and 


and 


u x - v v - VD 


x u y v — 

2 - <1- ^’ 


VD > U X + Vy — 2. 


D ^> | 2 U X Vy | t t D > (2 Zia; Uy) , 

which is an immediate consequence of 6 D after expanding the parentheses and 
collecting similar terms. 

Let us now show that the linear map 72(2) ■ U(z) —» 17(2), ||7?(2)|| < £ exists 
and is continuous with respect to 2. Consider the space £ of all continuous maps 
from 2 £ X into the ball with radius £ in the space L{U (2), V (2)) of linear maps 
from U(z) into V(z). The space £ is a complete metric space with respect to the 
standard supremum norm, because X is compact and L(U (2), V (2)) is complete. 
On the space £ we can define a map tp :£—>■£ by virtue of (H!21) , namely 

< P^(<h" 1 (2)) = -( 7 ( 2 ) - 5(2)72.(2)) (a(z) - /3(z)7Z(z)) \ 

The map *P is well-defined. Firstly because the image of the continuous map 7 Z 
is continuous. This follows because the operator (a(z) — /? (2)72(2)) is bounded 
away from the set of non-invertible operators by virtue of the choice of e. Sec¬ 
ondly, by virtue of |72(2)| < e we have |tp72(2)| < e, again by the choice of 
£. Moreover, the map tp is contracting for the same reasons. Hence the map 
*P has a unique fixed point 72q £ £, i.e., 72o(2) is the sought linear operator, 
continuously depending on 2 £ X. 

Thus at the points of the set X we have a continuous family of operators 
72o(2) such that y(2) = 0. Here ||5(2) -1 || < 1 and ||5(2)|| < 1. Replacing the 
subspaces U(z) = {u = 0} and V(2) = {u = 0} by 17(2) = (u = v+1Zq(z)u = 0} 
and 17(2) = 17(2), we obtain another continuous decomposition T^(Co D S) = 
U(z) ® 17(2). In this decomposition the matrix H -1 (2) has the form 


A ' 1 ® 


a- ( 2 ) /3(2) 
0 5(2) 


for all 2 in A'. 


Compactness of X implies the existence of constants u x > 1 and v y < 1 
such that ||5(2) -1 || < u and ||5(2)|| < v y . In the new decomposition we have 
u y = u v and v x = 0. Hence the conditions of (14T1) are satisfied also in the 
new decomposition. Applying the procedure above again to the operator A(z) 
in the new decomposition, we obtain the sought decomposition T?(Cq D S) = 


U(z) © y(20 


□ 


32 It can be shown that the decomposition T^(Co D«S) = U(z) ® V(z) is not only continuous 
but even Holder with respect to z £ X, but this is not relevant for the following argument. 
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Remark 17. Let us explain why the operator TZo(T) is in general not smooth. 
Equation (1421) is an equation on functions, it can be written in the simplest form 
as 


y(x) = f(x,y{<j){xj)}, 

where x is a coordinate, f(x,y) and <j>(x) given smooth functions, and y(x) the 
sought function. The function / satisfies the condition ||/'|| < 1. We may even 
assume that x and y are 1-dinrensional, this has no influence on the effects in 
connection with the smoothness of the solution. 

If there were no function </>, i.e., 4>(x) = x, then the relation would have 
the form y = f[x,y). In this case the implicit function theorem guarantees the 
existence of a smooth solution. Namely, the inclusion y £ C 1 , which implies 
also the inclusion y £ C°°, can be shown by the explicit formula 


y'{x) 


fL(x,y{x)) 

1 - fy(x,y(x))' 


Indeed, we obtain y' £ C 1 , this in turn yields y £ C 2 and so on. 

If <f>(x) is not the identity in x, but rather a map of some compact set 
on itself, then we can still show the existence and uniqueness of a continuous 
solution on this compact set, as was done in Lemma 1551 However, the solution 
of the functional equation need not to be smooth, just Holder. For example, the 
equation y(x) = ^y(3x ) has the solution y(x) = |x| Iog32 , which is not smooth 
at x = 0. 


We now consecutively apply the procedure described in Lemma [2H1 to the 
different decompositions of the tangent space. In this way we transform the 
(—7V)-th power = 0# of the differential of the Poincare map from Lemma 

1261 to the most convenient form. The following reasoning is almost identical 
for both sets E^^Xq) and Xjj(xq). We hence denote by X° any of the sets 
Z° H (x 0 ) H S H or X%(x 0 ) D S H . 

In order to bring the differential dQn{z) to a block-diagonal form, we first 
restrict our attention to the zero section Cq . Moreover, for the moment we 
shall not consider the coordinate w. Denote the subspace of T z Cq which cor¬ 
responds to the primary brackets by , and the subspace which corre¬ 

sponds to the secondary brackets by T}Cq . Decompose the subspace T°Cq 
by virtue of Lemma EB1 and substitution (1551) . such that T®Cq = U{z) © V(z) 
with dim U(z) = 1 . The operator A(z) becomes block-diagonal and expands 
U(z) and contracts V(z). Now we extend the subspace V(z) by adjoining the 
subspace T^Cq . In the obtained decomposition the operator dQn \t°c h ®t 1 c h 
has the form m with 7 (z) = 0 and || a(z) 1 || < 1. Since g(z) > 2 we can 
choose N large enough to enforce ||(5(z)|| < 1. The number N can be chosen 
independent of z by the compactness of the set X°. Therefore we can apply the 
procedure described in Lemma 1551 to dOff \t°c ha result, the operator 
dQH\ T K C H^fK C H becomes block-diagonal. Repeating this step, we consecu¬ 
tively adjoin the tangent subspaces corresponding to the variables w and y. to 
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the decomposition. The dimension of the subspace U(z) will increase by exactly 
1 at the last step, namely the adjoining of the coordinate fi. 

Thus we obtain a decomposition T Z C H fl S H = U(z) ® V{z), continuously 
depending on 2 £ A' 0 , such that dim 1 /( 2 ) = 2, and the subspace V(z) is hori¬ 
zontal, i.e., V(z) C T z Cq . Moreover, we have 

^-(t' «,>)• 

where a(z) : U(z) —> U(Qh{z)) and f3{z) : V(z) —> V(Qh{z)) depend continu¬ 
ously on 2 , and ||a( 2 ) _1 || < 1, ||/3(z)|| < 1. By the compactness of S^(xo) C\S H 
and Xfj(xo)r\S H we may assume that the norms of these operators are bounded 
away from 1, i.e., ||a( 2 ) _1 || < k and ||/3(;?)|| < k for some k < 1. 

Let us now apply the Hadamard-Perron theorem. To this end we construct 
a system of local coordinates in the neighbourhood of A 0 . Since the set A' 0 is 
compact, for every e > 0 there exists 5 > 0 such that for every point z 0 £ X° 
the difference &h{z) — g ? 0 #( 2 o )(2 — z 0 ) is bounded by e in the C l metric for 
every z from a (5-neighbourhood of zq. Let us cover A 0 with a finite num¬ 
ber of (5/2-neighbourhoods in Cq with centers 2?, ..., z° M £ X°. In the 6- 
neighbourhood of each point z „ we introduce local coordinates, defined by the 
block-diagonal structure of the differential d0ij(^). If the point z £ Ao lies in 
a 5/2-neighbourhood of 2 ^, then the norm of the difference <IQh{z) — d0_f/(2^J 
is bounded by e. If e is sufficiently small, then the Hadamard-Perron theorem 
is applicable to the orbit 0^(z), k £ Z. By virtue of this theorem, for each 
point zo £ A 0 there exists a smooth 2-dimensional manifold W + {zq) in the 5/2- 
neighbourhood of zq in C^nS H , consisting of points 2 which tend exponentially 
fast to the images of Zo under iterations of the map Qjj 1 - Since 0# = <I>7/ V and 
the set A 0 is compact, we have that every point 2 £ W + (z 0 ) tends to the image 
of 20 also under iterations of the map itself, 

W + ( 2 0 ) = j 2 £ C H nS H and \\z - 2 0 || < |, 

such that 11^(2) - $^( 2 0 )|| < CX k , k £ N }. 


Here the constants C > 0 and 0 < A < 1 from the Hadamard-Perron theorem 
depend only on the choice of n and e, and hence coincide for all points 2 . By 
virtue of compactness of the set A 0 we may choose constants C 1 > 0 and 
0 < Ai < 1 such that ||( 2 ) — <h^(2 0 )|| < C/A^ for k £ N. 

We shall need some properties of the constructed sets W + (zo)- 

(a) We shall now describe the trajectories of the original Hamiltonian system 
with Hamiltonian H which emanate from points in the set W+(2o), 20 £ A 0 . 
To this end we use the vector field h£h(x 0 ) on Ch in the neighbourhood 
of C/f, which has been constructed in Section RTTl cf. also system (ld2l) . Let 
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without loss of generality fi(z) > 0. The trajectory y(s) starting at the 
point 7 ( 0 ) = z £ W + {zq) intersects the surface S H of discontinuity of the 
right-hand side of the Hamiltonian system a countable number of times 
as s —> +00, namely at the points <f>^(z), k £ N. This trajectory exists 
and is unique, because the vector field ^h{x 0 ) is transversal to S H in a 
neighbourhood of X°. The time spent between two consecutive switchings 
is bounded by ^s m i n and 2s max from below and from above, respectively. 
Therefore y(s) tends to Cq = {^ = 0} exponentially fast as s —> +00, i.e., 

Cie -C2S < /r(y(s)) < c 3 e _C4S 

for some positive constants Ci, i = 1,... ,4. Since dt = fj,ds, we have that 
the integral 


r+00 

T(z) = / /i( 7 (s))ds < 00 

Jo 


is finite, and hence the preimage B _ ( 7 (t)) of the trajectory hits the point 
xo in finite time T(z). In other words, the trajectory X(t,z ), z £ M of 
the original Hamiltonian system with Hamiltonian H which starts at the 
point X( 0 ,z) = z £ B _ (W + (2:o)) exists and is unique in the interval 
t £ [0, T(B(i))]. Moreover, it hits xq at the moment T(B(2)). In the sequel 
we will omit the symbol B for brevity and write T(z) instead of T(B(Sf)). 

Note that the primary brackets IC m (X(t, z)) satisfy the inequalities claimed 
in Remark [13] 

(b) We shall now describe how two sets W + (z 0 ) and W + {z\ ), defined in 5/2- 
neighbourhoods of Zq and z 1 , may intersect. Let U denote the intersection 
of the 5/2-neighbourlroods of z 0 and z\. Then W + {zq) and W + (z\) either 
do not intersect, or coincide on U . We shall now prove this claim. If their 
intersection is not empty and contains some point z £ U, then the iterates 
of z tend to both the images of z0 and the images of z\. Therefore the 
images of z 0 and z\ approach each other exponentially fast. Consequently 
the iterates of an arbitrary point z £ W + (zo ) approach the iterates of z 1 
and vice versa. Therefore the sets W + (zi ), *=1,2 coincide on U. 


The sets S#(a; 0 ) and Xh(xq) have the following properties. Their intersec¬ 
tion with S H is precisely the union of all fibers B~ 1 (H /+ (zo)) over the points 
zo, 


Z H (xo )ns H = |J B-^W+^o)), 

z 0 GS 0 h (x 0 ) 

X H (x 0 )n S H = (J B-^w+izo)). 

zoex^(xo) 
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The sets are obtained by releasing a trajectory of the Hamiltonian system from 
every point in S H and continuing it up to the next intersection point with S H . 
Therefore properties (I) and (II) in both Theorem [7] and Theorem [5] follow from 
property (a) of the set W + {zq). 

The maps and to the topological Markov chains Ejt and which 
have been announced in (III), are obtained by virtue of the corresponding maps 
for model problem 0. namely as compositions of: (i) the projection 7rpp, which 
takes 2 £ B~ 1 (FF + (zo)) to the point zq £ Cq , (ii) identification of X^(xo) 
and S^(xo) with X + / g and E/g by means of the map n + /g from Lemma |24l 
(iii) application of the corresponding quotient map form model problem 0, 
and (iv) discarding some leading symbols in the obtained topological Markov 
chain. More precisely, for the map in (ii) we use the map 'Lp/g from 
Theorem[6l and for the map we use the map H/oi/<? from Theorem[2] Note 
that the projection 2 i —> zq itself is not well-defined, since it is possible that 
z £ B^ 1 (IT + (zo) (~1 W + (z i)). However, by virtue of property (b) the iterates of 
zq and Z\ approach each other exponentially fast in this case. Therefore there 
exists a number K £ N such that in the images ('Fp /<?) ((H_|_/<?) —1 (^i)) £ Ejt or 
(^oi/g)((n+/g){zi)) £ Eoi differences can appear only at positions with indices 
smaller than K. At positions with numbers larger than K these images have 
to coincide. By discarding all symbols with indices smaller than K (denote this 
operatoi^l by 5) we hence eliminate the ambiguity in the map ^xw ■ Therefore 

= 5 o (^r/a) ° ( n +/5) 1 ° *w 

= 5 ° (^oi/ff) ° ( n +/g )~ 1 ° kw 

This proves claim (III) for both Theorems [7] and [HJ 

Claim (IV) of Theorem [8] is proven by replacing the set Xq by a finite set, 
consisting of the intersections of Rijk/g or Qi/g with S (or more precisely, of 
their images under the map n +/g). In the first case the set Xg consists of 
three points, in the second case of four points. In the case of the three-link 
chain all eigenvalues of the operator A 3 from (Idfll) have a modulus larger than 
1 (see Section 17751) . Hence we obtain in this case that the manifold W + is 1- 
dimensional. For the case of the four-link chain the operator A 4 has exactly 
one eigenvalue with modulus smaller 1 (see Section [2781) . Hence in this case the 
manifold W + is 2-dimensional. 

Item (IV) from Theorem|7]on the bounds on the dimension of Xh(xq) follows 
from the corresponding item in Theorem [G] Indeed, for some sufficiently small 
/xo the map B -1 is a diffeomorphism on Cq x {0 < /i < g, o} and does not 
change dimensions. Secondly, if dG(x o) = 0, then the dimensions (Hausdorff or 
Kolmogorov) of the sets X^(xq) and X/g coincide, because in this case the map 
n+ is locally Lipscliitz by virtue of Lemma EH It remains to note that the sets 

33 For sequences in Ejt, which are infinite only to the right, we discard the K first symbols, 
and for sequences in Eoi, which are bilaterally infinite, we discard the whole infinite sequence 
of symbols left of position K. 
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Xh(x o) and X + are obtained from the sets X^(x o) D S H and (X fl S)/g by a 
similar procedure, namely two-dimensional stable submanifolds are constructed 
and trajectories of the system released. 

Item (V) of Theorem [7] is identical with the corresponding item in Theorem 

EE 

Items (VI) of Theorem [7] and (V) of Theorem [5] can be obtained by consid¬ 
ering the Lipschitz surface M_ instead of the Lipscliitz surface M + of optimal 
trajectories. Recall that M_ consists of trajectories which leave the origin of 
Hamiltonian system ([3]) determined by the Pontryagin maximum principle for 
model problem HD- This completes the proof of Theorems [7] and [51 


□ 

Let us now prove the claim of Remark 1 131 The upper bound has been already 
obtained in Lemma 1231 The lower bound is equivalent to the assertion that on 
every trajectory X(t) from the set Xh(xq) or E.h{xo) we have the bound 

n{X(t))>d'{T-t), 

where c" > 0 is some constant, and T is the time instant of hitting the strange 
point xq. 

Let us transfer the trajectory X(t) to the cylinder Ch by means of the blow¬ 
up map B and perform the substitution ds = -jjdt of the time parameter. Then 

we obtain a trajectory X(s) = B(X(t(s)) of system (l34l) . Hence we have to 
show that 


__ r+oo ~ 

/u.(X(s)) >T t(s) = c" / fj,(X(a)) da. 

J S 


Denote by X°(s) the trajectory on Cq which emanates from Xh{x o) or 
’EP h (xo) which the trajectory X(s) converges to as s —> +oo. As we have men¬ 
tioned above, the trajectory A'°(s) is also a trajectory of system (1371) . since we 
have fj,(X°(s)) = 0. Consider also some trajectory A 1 (s) of system (1571) which 
differs from X °(s) only in the coordinate /x, say /x(X 1 (s)) > 0, such that 


f K(X\s)) 

I 

(*» 

{ rtV'(O)) 


K(X°(s)), 

w(X°(s)), 

^X\s))T(X\s)) = ^X\s))T(X°(s)), 


M (X(0)). 


Since the trajectory A' 1 (s) is the image of an optimal trajectory for model 
problem (JTJ) under the map B on + , we have by virtue of TheoreniQEthe following 
inequality: 


_ r +oo __ 

H{X 1 {s))>c o n{X 1 (a))da 

J S 
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on this trajectory for some constant cq > 0 . 

Let us now show that the values of /i do not differ too much on both trajec¬ 
tories X(s) and X 1 (s), in the sense that 

f £h{X{8)) = »{X{8))T(X{8)), 

{ = n( x \s))T(X\s)), 

and 

■f In M 5 (5)) = T(X(s)) - T (X\s)). 

Since both X(s) and -Y 1 (s) tend to X°(s) exponentially fast, the difference at 
the right-hand side in the last equation above also tends to 0 exponentially fast. 
Therefore we have for some constants C\ , ci > 0 that 


—cie 


-C- 2 .S 


< li n igi 

" ds »(Xi(s)) 


< c ie 


— C2S 


Taking into account that In = 0, we immediately obtain that for C 3 = 

e Cl / C2 we have 


< »( x ( s )) < cs/rp^s))- 

C3 

Thus 


~ 1 ~ r r+°° _ r r+00 __ 

K x (s)) > —M( xl (s)) > — / n(X 1 (a))da > 4 / h( x (<j)) da, 

C 3 c 3 J s eg 7 S 

what is what we had to show. 

Remark 18. By virtue of Remark [13] we can control how the sets S#(a;o) and 
Xh(x 0 ) enter the strange point Xq. In geometric terms the sets S^po) and 
Xh{x 0 ) are tangent (in the sense of the order in fi) to the cone B _1 (V Q x{^£ 
R}) . 

Remark 19. By Definition [9] a strange point can appear only if the number of 
degrees of freedom of the system is not smaller than 16. Indeed, in Definition 
1 we demand the linear independence of the differentials of the functions F r , 
(ad Fi)F r etc. up to the 4-th order. The minimal dimension where such a 
situation can occur can be easily computed by means of the dimension of the 
free nilpotent graded Lie algebra of depth 4 and with three generators (F 3 has 
to be excluded, since F$ = —F\ — F 2 ). Namely, the growth vector of such an 
algebra can be calculated via Halls words, it equals (3,6,14,32). Since we do 
not use dFo at the first level, the minimally possible dimension of M is 31. But 
dimAI must be even, hence dirnAI > 32 and the system has at least 16 degrees 
of freedom. However, the conditions for the definition of a strange point are 


111 





excessively strong, and that is why the number of degrees of freedom is so large. 
In fact, the conditions can be substantially weakened. Hamiltonian system ©, 
which is determined by the Pontryagin maximum principle for model problem 
m, is a low-dimensional example (with 4 degrees of freedom) of a Hamiltonian 
system which exhibits the phenomenon described in Theorem 0 

Remark 20. The appearance of strange points in large dimensions is not avoid¬ 
able and they cannot be removed by a small perturbation of the Hamiltonians 
Hi, i = 1,2,3 if we assume that the gradients of the commutators at Xq are 
linearly independent not only up to the 4-th order, as demanded in Definition 
1 but up to 5-th order. In this case the set ST of strange points is a smooth 
manifold in the neighbourhood of xq and its codimension can be computed by 
means of the dimension of the free nilpotent graded Lie algebra of depth 5 
and with three generators. The growth vector of such an algebra is given by 
(3,6,14,32,80). The codimension of ST is smaller than 80 by a number of 
4. Here 3 dimensions come from the symmetric form B rr i and one from the 
function To at the first level. Hence we get 

codim ST = 76. 
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